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INTRODUCTION 


This  report  summarizes  the  work  of  the  Collaborative  Design  Technology  Laboratory  (CDT 
lab)  -  a  component  of  the  Design  Technology  Branch,  Armstrong  Laboratoty  Human 
Engineering  Division  (AL/CFHD),  at  Wright-Patterson  Air  Force  Base  (WPAFB)  in  Dayton, 
Ohio.  Earlier  work  under  the  project  name  AKADAM  (Advanced  Knowledge  and  Design 
Acquisition  Methodology)  had  concentrated  on  IT-supported  knowledge  elicitation  in  the  service  of 
systems  design,  organizational  re-design,  and  total  quality  management  (cf.  McNeese  et  al,  in 
press).  The  CDT  Lab  was  established  as  a  follow-on  extension  of  the  AKADAM  work  (and 
primary  AKADAM  researchers)  aimed  at  addressing  collaboration  in  design  and  its  facilitation 
through  information  technology.  This  report  will  summarize  the  work  of  the  Collaborative  Design 
Technology  Team  (CDTeam)  during  the  period  1993-1995. 

The  CDT  Lab  was  a  component  of  AL/CFHD  work  aimed  at  realizing  Computer  Aided 
Systems  Human  Engineering  (CASHE),  as  described  in  Boff  et  al.  (1991).  Broadly  speaking  the 
CASHE  "vision"  entailed  bringing  advanced  information  technology  (IT)  to  bear  on  the  task 
domain  of  human  factors  engineering  in  systems  of  potentially  large  scale  and  complexity,  as 
illustrated  in  Figure  1.  The  general  goal  was  to  promote  ergonomics  /  human  factors  as  a  "full 
partner"  among  the  participants  in  systems  design.  The  more  specific  goal  was  to  develop  and 
deploy  tools  supporting  human  factors  professionals  in  a  manner  analogous  to  the  way  CAD 
systems  support  structural  engineers  or  CASE  tools  support  software  programmers. 


Figure  1:  The  CASHEVision 

Figure  1  shows  a  multidisciplinary  design  team  working  with  a  CAD/CAE  system  and  various 
design  suonort  tools  to  assess  human  oerformance  implications  of  design  decisions.  The 


combination  of  integrated  visual,  audio,  and  virtual  reality  display  technologies  promotes 
immediacy  and  fidelity  in  portraying  outcomes  of  crew  system  design  decisions,  even  during  early 
conceptualization.  All  participants  (e.g.,  program  managers,  engineers,  end  users),  can  see  how 
the  potential  design  changes  will  affect  (e.g.)  product  form,  potential  functionality,  project 
scheduling,  budget  criteria,  etc.  By  linking  the  illustrated  meeting  scenario  to  other  such  sites, 
team  members  can  readily  and  reliably  communicate  design  concerns,  proposals,  and  solutions  to 
others  throughout  a  distributed  network.  These  capacities  are  of  particular  relevance  to  U.S.  Air 
Force  systems  design  and  acquisition  efforts,  which  typically  involve  hundreds  of  organizations 
and  thousands  of  people  distributed  across  North  America. 

The  Collaborative  Design  Technology  Laboratory  (CDT  Lab)  was  established  in  summer  1993 
as  AL/CFH's  work  unit  dedicated  to  research  and  development  in  the  area  of  applying  information 
technology  (IT)  in  support  of  design  teams.  The  CDT  Lab's  mission  as  specified  for  the  Logicon 
Technical  Services  Inc.  support  contract  to  AL/CFH  is  "...  to  apply  a  user-centered  approach  to  the 
development  of  collaborative  design  technologies.  Through  the  study  of  multidisciplinary  design 
teams,  the  evolving  collaborative  technologies  can  be  made  to  reflect  the  needs,  capabilities,  and 
limitations  of  the  users."  More  concretely,  the  laboratory's  mission  was  defined  by  Human 
Engineering  Division  management  in  August  1993  as:  “...  (to)  enable  and  facilitate  distributed 
group  decision  making,  problem  solving,  and  ‘concept  visualization’  for  simultaneous  engineering 
and  design.” 

The  CDT  Lab  has  pursued  its  mission  by  exploring  the  applicability  of  group  support 
technologies  to  USAF  design  /  acquisition.  This  focus  has  been  motivated  largely  by  the  fact  that 
design  /  acquisition  accounts  for  the  majority  of  all  cost  burdens  (time,  money,  and  human 
resources)  necessary  to  deploy  complex  systems  (e.g.,  aircraft)  in  support  of  USAF  missions. 
This  focus  is  further  justified  by  CDT  Lab's  location  at  WPAFB  —  the  primary  site  for  the  Air 
Force  Materiel  Command,  the  Aeronautical  Systems  Center,  and  the  System  Project  Offices 
(SPO's)  overseeing  USAF  design/acquisition  for  major  systems.  The  June  1994  DOD  Detailed 
Technology  Area  Plan  (DTAP)  on  Human  Systems  Interface  details  the  goal  of  its  Distributed 
Collaboration  Technology  Effort  (which  includes  the  CDT  Lab)  as  being  "...to  develop  methods  to 
reduce  collaborative  planning  time  by  25  percent  and  collaborative  execution  time  by  10  percent." 
By  pursuing  the  DTAP  goals  with  respect  to  the  area  of  highest  USAF  cost  burden  in  system 
deployment  (design/acquisition),  the  CDT  Lab  has  aimed  to  translate  these  percentage  targets  into 
maximum  budgetary  savings. 

The  focal  points  for  such  work  are  evident  in  Figure  1  —  a  computer-supported  co-located 
meeting  linked  via  teleconferencing  to  remote  partners.  The  specific  human  engineering  orientation 
to  systems  design,  coupled  with  the  situation  of  CDT  Lab  within  a  major  human  factors 
organization,  resulted  in  CDT  Lab's  common  involvement  in  human  factors  aspects  of  design 
efforts.  However,  the  issues  and  experiences  deriving  from  these  human  engineering  efforts  have 
an  applicability  beyond  the  specific  field  of  human  factors.  Large-scale  systems  design  is  today 
accomplished  by  multidisciplinary  design  teams  drawn  from  a  variety  of  backgrounds  and 
organizations.  Examples  of  other  (not  exclusively  humein  factors)  target  scenarios  involved  in 
large  systems  design  and  acquisition  include: 

•  User-centered  /  Participatory  design  exercises,  wherein  design  professionals 

must  interact  with  (e.g.)  organizational  managers  and  end  users. 

•  Knowledge  elicitation  exercises,  where  by  definition  the  sources  of 

"knowledge"  (experts)  are  identified  with  respect  to  differential  skills  or 

expertise. 

•  Educational /Training  meetings,  in  which  (e.g.)  end  users  are  familiarized  with 

the  emerging  product. 

•  Usability  testing  exercises,  in  which  developers,  testers,  and  end  users  assess 

the  emerging  product 
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•  Project  management  meetings. 

Beyond  design  and  acquisition,  the  USAF  has  an  interest  in  optimizing  collaboration  in 
operational  areas  such  as  multi-operator  crew  systems,  command  and  control  systems,  logistics, 
planning,  and  the  organizational  re-engineering  aspects  of  ongoing  DOD  adjustments  to  the  post- 
Cold  War  era.  Owing  to  this  broad  scope  of  applicability,  the  CDT  research  agenda  was  framed 
without  specific  regard  to  supporting  human  factors  or  ergonomics  professionals  per  se. 

More  specifically,  the  August  1993  mission  statement  laid  out  the  following  four  task  elements; 

1)  Develop  and  evaluate  procedures  and  protocols  optimized  to  state-of-the-art  media. 

2)  Develop  innovative  group-human  system  concepts. 

3)  Model  and  simulate  advanced  groupware  to  assess  human  and  technology  demands 

and  implementation  feasibilities. 

4)  Transfer  emergent  group  collaboration  technical  capabilities  to  industry. 

The  first  task  element  explicitly  linked  our  work  to  the  state  of  the  art  in  the  communication 
media  underlying  collaborative  applications  of  information  technology  (IT).  Based  on  the  term 
"innovative"  connoting  the  ability  to  surpass  the  status  quo,  the  second  task  element  explicitly 
linked  our  work  to  the  state  of  the  art  in  human  computer  interaction  (HCI)  as  it  applies  to  group  IT 
usage.  The  third  task  element  explicitly  linked  CDT  Lab  activities  to  the  current  state  of  research 
and  commercialization  in  the  research  area  termed  computer  supported  cooperative  work  ( CSCW) 
and  those  IT  applications  for  team  support  termed  groupware.  The  remainder  of  this  report  will 
present  the  activities  related  to  these  three  goals.  The  issues  and  experiences  relevant  to  these  three 
task  elements  have  an  applicability  to  a  variety  of  activities  not  necessarily  subsumed  within 
"design"  —  e.g.,  management  decision  making,  group  knowledge  elicitation,  or  collaborative 
document  writing.  As  a  result,  the  CDT  research  agenda  was  framed  with  respect  to  the  demands 
of  collaboration  on  information  technology  generally.  The  fourth  element  —  external  technology 
transfer  --  lies  outside  the  scope  of  this  report. 

These  goals'  allusions  to  "state-of-the-art,"  "innovation,”  and  "advanced"  all  connoted  that  we 
be  aware  of  the  status  of  research  and  development  in  the  relevant  areas.  We  therefore  initially 
prioritized  comprehensive  assessment  of  the  “state  of  the  art”  in  terms  of  research,  concepts,  tools, 
and  products  across  the  R  &  D  fields  targeted  by  the  task  elements.  Some  of  the  tactics  employed 
in  this  assessment  included:  literature  reviews;  vendor  contacts;  monitoring  on-line  news  groups; 
and  collecting  materials  from  Internet  sources.  A  large  compendium  of  product  information, 
reviews,  academic  papers,  and  further  contact  information  was  assembled  and  continually  updated. 
Selected  interactive  resources  we  identified  (primarily  information  sites  on  the  Internet  /  World 
Wide  Web)  are  listed  in  Appendix  A. 

The  second  task  element's  focus  on  "advanced  group-human  systems  concepts"  led  us  to 
critically  re-evaluate  the  state  of  the  art  in  group  support  tools  and  devise  a  new  strategy  for  fitting 
such  tools  to  their  sets  of  users.  The  results  of  these  efforts  will  be  discussed  later  in  this  report 
with  reference  to  the  Group  Interface  (GI)  and  the  Unified  Interface  Surface  (UIS)  prototype. 
More  detailed  discussion  of  these  efforts  will  be  found  in  a  companion  Technical  Report 
(Whitaker,  Longinow,  &  McNeese,  1995).  Finally,  we  conducted  studies  of  human  factors  in 
group  design  processes  to  provide  background  for  our  evaluation  of  state-of-the-art  media  and 
formulation  of  advanced  group-system  concepts.  Examples  of  these  research  efforts  are  described 
later  in  this  report  and  in  other  publications  as  noted. 
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THE  SCOPE  OF  CDT  ENQUIRY;  THE  CDT  "TRIANGLE 


The  set  of  research  and  development  areas  mentioned  as  relevant  to  CDT  enquiry  was 
enormous,  owing  to  the  topical  and  administrative  links  to  AKADAM,  CSCW,  remote 
teleconferencing,  knowledge  elicitation,  user-centered  or  participatory  design  practice,  the  CASHE 
"vision,"  and  our  diverse  sibling  project  groups  within  the  Design  Technology  Branch.  Planning  a 
specific  research  program  required  us  to  (1)  circumscribe  our  research  "territory"  and  then  (2)  chart 
a  progressive  path  of  activities  through  it.  We  found  the  most  useful  device  in  achieving  the  initial 
circumscription  of  our  scope  to  be  the  "CDT  Triangle"  --  a  diagram  admittedly  playing  on  our  own 
name  —  illustrated  in  Figure  2. 

The  triadic  relationship  between  collaboration  (an  interactive  group  process),  design  (a 
constructive  task),  and  technology  (specifically  information  technology)  proved  very  helpful  in 
formulating  the  CDTeam's  agenda.  The  links  between  the  three  nodes  succinctly  captured  the  trio 
of  topical  areas  upon  which  the  CDT  Lab  work  could  focus:  Collaborative  Design  (CD); 
Collaborative  Technology  ( CT);  and  Design  Technology  (DT).  Our  interpretation  of  the  CDT  Lab 
mission  was  to  conduct  basic  and  applied  research  addressing  Collaborative  Design  (CD)  and 
Collaborative  Technology  (CT)  in  preparation  for  leveraging  advances  in  Design  Technology  (DT). 

As  a  result,  we  framed  our  initial  agenda  to  address  the  CD  and  CT  "legs"  of  the  CDT  Triangle, 
and  it  will  be  work  in  these  directions  that  is  summarized  in  the  balance  of  this  report.  The  next 
two  sections  will  address  Collaboration  /  Collaborative  Design  and  Collaborative  Technology, 
respectively,  to  introduce  the  reader  to  the  topical  background  and  issues  of  currency  which  drove 
our  research  during  the  period  1993-1995.  Subsequent  sections  will  detail  the  CDT  Lab's 
infrastructure,  approaches  to  these  issues,  and  specific  research  activities. 


design _ TECHNOLOGY 

Figure  2:  The  CDT  "Triangle"  configuration  of  research  issues 
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Collaboration  /  Collaborative  Design 


The  CDT  research  agenda  was  tailored  to  address  issues  of  collaboration  in  design.  The 
motivations  for  this  programmatic  direction  lay  in  current  events  within  the  United  States  Air  Force 
specifically  and  the  design  community  at  large.  Design  practice  had  been  shifting  more  and  more 
toward  Concurrent  Engineering  ( CE),  which  the  Institute  for  Defense  Analyses  Report  IDA  R-388 
defined  as  “...a  systematic  approach  to  the  integrated,  concurrent  design  of  products  and  their 
related  processes,  including  manufacturing  and  support.”  Concurrent  Engineering  approaches  to 
design  mandate  that  all  parties  to  the  generation  of  a  given  end  product  (e.g.,  designers,  end  users, 
manufacturers,  marketers)  be  actively  involved  throughout  its  design,  development,  and 
deployment.  In  other  words,  the  diverse  participants  in  product  development  work  together  in 
tandem  during  the  whole  process,  rather  than  each  working  his  or  her  "little  part"  during  some  part 
of  the  overall  process. 

The  novelty  in  CE  is  well  illustrated  by  analogy  to  automobile  manufacturing  (cf.  Whitaker  & 
Essler,  1990).  Henry  Ford's  inception  of  the  assembly  line  was  based  on  each  worker  adding  to 
the  emerging  automobile  in  a  stepwise  fashion,  then  letting  the  subassembly  continue  down  the  line 
to  the  next  station.  Although  all  the  workers  on  the  line  could  be  seen  as  collectively  assembling 
the  car,  each  station  along  the  way  was  functionally  specialized  to  the  point  that  the  only  real 
"collaboration"  among  the  workers  was  to  accept  and  pass  on  the  emerging  product.  In  contrast, 
team-based  automobile  assembly  at  Volvo's  Kalmar  plant  in  Sweden  was  based  on  a  small, 
persistent  team  of  workers  assembling  a  single  car  from  start  to  finish.  Conventional  approaches 
to  large  systems  design  followed  the  Ford  model  -  linear,  stepwise  progression  from  design 
through  manufacture  to  test  and  deployment.  Concurrent  engineering  takes  the  Kalmar  approach  ~ 
bringing  together  a  team  whose  members'  respective  expertise  will  be  applied  in  concert  with  their 
peers  throughout  the  entirety  of  the  production  process. 

In  1993,  the  U.S.  Air  Force  officially  adopted  Concurrent  Engineering  as  Integrated  Product 
Development  (IPD),  which  Major  General  James  Fain,  then  Commander  of  the  USAF 
Aeronautical  Systems  Center,  defined  as  “a  philosophy  that  systematically  employs  a  teaming  of 
functional  disciplines  to  integrate  and  concurrently  apply  all  necessary  resources  to  prepuce  an 
effective  and  efficient  product  that  satisfies  customers’  needs.”  IPD  was  to  be  accomplished  by 
multidisciplinary  Integrated  Product  Teams  (IPT's)  working  in  tandem.  The  USAF  adoption  of 
IPD  provided  CDTeam  with  a  ready-made  focus  for  its  work  on  Collaborative  Design. 

The  groundwork  for  such  work  had  already  been  laid  in  a  study  involving  design  professionals 
from  the  USAF's  Aeronautical  Systems  Center  (ASC),  as  described  in  McNeese  et  al  (1993).  A 
panel  elicited  knowledge  from  each  of  seven  experienced  human  factors  specialists  using  the 
concept  mapping  technique.  Each  specialist  was  asked  to  recount  actual  experiences,  and  a  concept 
map  was  constructed  to  leflect  these  experiences  plus  any  generalizations  or  insights.  These 
individual  reports  were  then  compiled  into  a  summary  map  outlining  the  general  issues  and  insights 
discernible  in  the  entire  set.  Although  no  detailed  topical  focus  had  been  planned  or  related  to  the 
specialists,  the  resulting  summary  map  concentrated  on  specific  aspects  of  design  teams’ 
organizational  structure,  interpersonal  relationships,  and  the  influence  of  these  on  the  resolution  of 
design  problems.  This  summary  map  portrayed  an  inventory  of  contrasts  between  traditional 
design  /  acquisition  groups  and  IPT's.  In  comparison  with  the  newer  IPT's,  subjects  characterized 
the  traditional  USAF  design  acquisition  groups  as  displaying: 

•  more  formal  conununication  structures  (e.g..  Critical  Design  Reviews) 

•  narrower  bandwidth  of  information  exchange 

•  a  more  strictly  evaluator/monitor  role  for  government  representatives 

•  strict  governance  by  the  systems  acquisition  development  procedures 

•  emphasis  on  a  'standard'  against  which  contractors  must  'get  it  right' 

•  strong  influence  of  personalty  factors 
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•  frequent  challenges  of  expertise  due  to  insufficient  understanding  of  others' 

perspectives 

In  contrast  with  the  traditional  design  /  acquisition  groups,  the  subjects  characterized  USAF 
IPT's  as  having: 

•  less  formal,  more  frequent  person-to-person  interaction 

•  less  commitment  to  forcing  black  and  white  decisions  regarding  gray  situations 

•  fewer  and  less  severe  design  bottlenecks 

•  increased  sense  of  teamness 

•  increased  levels  of  accountability 

•  increased  awareness  of  other  disciplines’  design  constraints,  making  design 

tradeoffs  less  obscure 

•  lessened  adversarial  relationships  between  government  and  contractor 
participants 

•  more  impact  on  actual  human  factors  engineering  designs 

Overall,  the  IPD  approach  was  believed  to  expedite  issue  resolution  and  to  promote  both  earlier 
and  better  integration  of  design  requirements  from  multiple  disciplines.  CDTeam  predicated  its 
research  on  the  thesis  that  the  key  factors  in  realizing  IPD's  apparent  benefits  were:  promotion  of 
intoteam  communications,  promotion  of  "flatter"  team  hierarchies  (i.e.,  a  more  "peer-to-peer" 
orientation),  and  promotion  of  information  sharing  throughout  the  process.  These  key  factors 
were  repeatedly  cited  in  the  general  literature  and  success  stories  in  Concurrent  Engineering  at 
large.  This  emphasis  on  interaction  and  information  sharing  was  also  reflected  in  the  best  available 
data  on  engineers'  problems  with  CE  practices.  In  a  1993  survey  of  engineers  (Bulleley,  1993), 
61%  of  responding  engineers  acknowledged  their  firms  had  gone  to  concurrent  engineering,  and 
over  half  of  them  claimed  CE  was  causing  them  more  stress  than  prior  practices.  This  result 
motivated  a  closer  inspection  of  the  survey  results  by  CDTeam.  The  respondents'  five  most 
frequently  cited  CE  drawbacks  were: 

1)  "too  many  meetings"  (52%  of  respondents) 

2)  "too  many  cooks  in  the  kitchen"  (38%  of  respondents) 

3)  "not  enough  design  time"  (32%  of  respondents) 

4)  "too  many  compromises"  (23%  of  respondents) 

5)  "emphasizes  social  skills"  (18%  of  respondents) 

The  “Top  5”  CE  drawbacks  reported  by  engineers  can  all  be  seen  as  problems  relating  to  the 
team  meetings  upon  which  CE  practice  relies.  The  first  and  third  most  frequent  complaints  entail  a 
problematical  tradeoff  between  meeting  participation  and  the  sort  of  design  work  previously 
conducted  individually.  This  implies  CE  meetings  are  inefficient  in  terms  of  (e.g.):  managing  time 
required  for  “synchronization’’  of  agendas;  managing  time  required  for  "coordination"  of 
viewpoints;  maintaining  consistency  across  sessions;  managing  time  required  for  meeting 
preparation;  and  managing  time  required  for  meeting  conduct.  We  concluded  that  integrating 
collaborative  technology  into  existing  work  formats  and  across  geographical  /  temporal  boundaries 
(e.g.,  via  wide-area  networks)  could  help  in  overcoming  these  efficiency  problems. 

The  second,  fourth,  and  fifth  most  frequent  complaints  entail  a  problematic  tradeoff  between 
designers'  technical  and  social  skills.  These  imply  CE  meetings  are  ineffective  in  terms  of  (e.g.): 
presenting  multiple  perspectives  on  the  design  problem;  understanding  and  appreciating  each 
others’  concerns;  and  formulating  consensus.  This  view  is  supported  by  experience  from  the 
CSCW  community,  where  up  to  80%  of  meeting  time  is  estimated  to  be  spent  on  such  cross¬ 
orientation  among  interactors  (Fuller,  1993).  We  concluded  that  collaborative  technology  support 
for  depicting,  manipulating,  and  contrasting  the  disparate  perspectives  of  multidisciplinary  design 
team  members  would  address  the  effectiveness  problem. 
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Because  the  CDT  Lab  was  a  follow-on  to  the  previous  AKADAM  work,  much  of  this 
delineation  was  framed  with  regard  to  the  issues  proven  to  be  critical  in  that  earlier  project.  The 
AKADAM  experience  highlighted  the  advantages  of  systems  design  being  user-centered,  and  it 
demonstrated  that  user-centeredness  could  be  achieved  in  practice  through  promoting  user 
participation  (McNeese  etal,  1993).  Most  of  the  AKADAM  team's  design  and  re-engineering 
support  activities  elicited  knowledge  from  individual  subject  matter  experts  (SME's),  but  others 
involved  sessions  in  which  groups  of  SME's  participated.  The  AKADAM  project  demonstrated 
that  its  knowledge  elicitation  techniques  could  support  teams,  but  those  aspects  of  knowledge 
elicitation  peculiar  to  team  applications  were  not  the  foci  of  the  research  per  se.  The  CDTeam 
accordingly  switched  from  devising  and  demonstrating  knowledge  elicitation  methods  for  groups 
(among  others)  to  researching  fundamental  issues  of  how  information  technology  and  structured 
group  processes  (such  as  the  AKADAM  methodology)  could  support  design  collaboration  in 
concurrent  engineering.  The  next  section  introduces  the  specific  topical  intersection  of  our 
knowledge  elicitation  expertise,  current  design  practice,  and  CSCW  research  —  design  rationale. 

Design  Rationale:  Knowledge  Construction  in  Collaborative  Design 

Design  decision  making  can  be  complex,  and  this  complexity  naturally  rises  in  proportion  to  the 
complexity  of  the  system  being  designed.  What  is  not  so  obvious  is  that  complexity  in  design 
decision  making  can  vary  along  another  dimension,  proportional  to  the  number  of  communicational 
and  interpretive  "passages"  that  must  be  traversed  in  the  course  of  achieving  final  consensus  on 
(e.g.)  requirements  specifications.  In  multidisciplinary  design  teams,  there  are  a  number  of  such 
"passages"  to  be  traversed  when  (e.g.):  comparing  disparate  frames  of  reference;  educating 
partners  about  one's  own  terminology  and  criteria;  explaining  conclusions  which  are  not  obvious 
to  anyone  outside  one's  own  specialty,  and  the  like.  Phrased  another  way,  a  good  deal  of  the 
work  lies  in  formulating  the  background  for  decision  making,  not  just  in  the  final  selection  of  one 
or  more  alternatives.  This  background  is  itself  a  form  of  expressible  "knowledge"  in  the  same 
sense  addressed  in  artificial  intelligence  (AI)  or  knowledge  acquisition  -  a  structured  model  with 
its  own  denotational  and/or  procedural  semantics.  In  this  case,  the  model  augments  knowledge  of 
the  task  domain  (e.g.,  a  task  analysis)  and  the  emerging  artifact  (e.g.,  requirements  specifications) 
with  knowledge  of  the  design  decision  making  process  as  well  as  the  intermediate  design  decisions 
made.  Such  a  model  of  the  process,  the  debate,  and  the  justifications  leading  to  a  particular  design 
is  termed  design  rationale,  which  has  been  defined  as: 

•  "...  the  design  problems,  alternative  resolutions...,  tradeoff  analysis  among 

these  alternatives,  and  a  record  of  the  tentative  and  firm  commitments  that 
were  made  as  the  problem  was  discussed  and  resolved."  (Conklin  & 
Begeman,  1988,  p.  304);  or 

•  "...a  historical  record  of  the  reasons  for  the  choice  of  an  artifact  ...,  a  set  of 

psychological  claims  embodied  by  an  artifact  ...,  and  a  description  of  the 
design  space  ..."  (Lee  &  Lai,  1990,  p.  4) 

Some  IT  tools  have  been  developed  to  provide  a  depictive  framework  within  which  design 
rationale  is  displayed  and  manipulated.  The  most  widely  known  such  application  is  gIBIS 
(Conklin  &  Begeman,  1988;  Burgess- Yakemovic  &  Conldin,  1990)  —  a  multi-user  hypertext 
system  developed  at  MCC  as  a  computer  implementation  of  Horst  Rittel's  IBIS  (Issue-Based 
Information  System)  planning  and  design  method  (Kunz  &  Rittel,1970;  Rittel,  1980).  The  gIBIS 
tool  consists  of  a  graphical  interface  allowing  users  to  address  and  manipulate  a  common 
representation  of  their  design  discussion,  based  on  these  basic  units  and  a  set  of  standard  relations. 
The  representational  schema  employed  has  three  main  components.  Issues  depict  any  topic  of 
discussion.  A  position  is  any  expression  which  addresses,  qualifies,  or  otherwise  informs  a  given 
issue.  Finally,  an  argument  is  any  expression  which  (as  a  propositional  unit)  either  supports  or 
raises  objection(s)  to  a  given  position.  This  reliance  on  a  schematic  representation  is  another  link 


7 


between  design  rationale  and  the  sort  of  knowledge  acquisition  practiced  in  AKADAM. 

Besides  gIBIS,  other  design  rationale  systems  include:  rIBIS  (Rein  &  Ellis,  1991)  —  a  real¬ 
time  version  of  gIBIS;  ArgNoter  (Foster  &  Stefik,  1986;  Stefik  et  al,  1987)  -  a  graphical  IT  tool 
for  displaying  positions  and  arguments  in  a  structured  fashion;  SYNVIEW  (Lowe,  1986)  —  a 
distributed  conferencing  /  decision  support  tool;  PHI  (McCall,  1987)  —  an  extension  of  the  IBIS 
argumentation  model;  and  JANUS  (Fischer  et  al,  1989)  —  an  application  of  PHI  to  reflect 
decisions  made  in  kitchen  design.  SIBYL  (Lee,  1990a;  1990b;  Lee  &  Lai,  1990)  is  a  more  highly 
structured  extension  of  gIBIS  which  adds:  (1)  a  more  formal  representation  language  (DRL  — 
Decision  Representation  Language)  and  (2)  a  decision  matrix  —  a  2-dimensional  grid  mapping 
alternative  positions  onto  specific  goal  states.  The  most  recent  development  in  the  gIBIS  lineage  is 
the  introduction  of  an  Microsoft  Windows-based  gIBIS  tool  called  CM/1  in  1994.  This  PC 
application  provides  the  functionality  of  the  original  gIBIS  in  a  commercial  package  with  a  graphic 
interface. 

As  a  form  of  knowledge  (in  the  Al  sense),  design  rationale  serves  a  number  of  useful 
functions.  It  records  design  decisions,  enhances  "organizational  memory,"  informs  subsequent 
decision  making,  and  explains  prior  design  decisions.  Design  rationale  is  most  critical  for  people 
who  must  coordinate  plans  across  distance,  coordinate  plans  over  time,  and  explain  or  justify 
intermediate  and  final  results.  Design  rationale  is  most  critical  for  systems  which  are  large, 
complex  in  form,  complex  in  function,  and  long-lived  in  service.  In  terms  of  both  people  and 
systems,  design  rationale's  highest  criticality  matches  the  profile  of  USAF  operations.  We 
therefore  elected  to  prioritize  design  rationale  as  a  focus  for  our  enquiries  into  collaborative  design. 
As  a  form  of  knowledge  (in  the  Al  sense),  design  rationale  is  amenable  to  elicitation,  depiction, 
analysis,  and  inferential  manipulation  as  done  in  (e.g.)  ATs  "knowledge  engineering".  This  point 
of  intersection  meant  that  our  team's  AKADAM  experiences,  techniques,  and  tools  could  be 
applied  to  design  rationale,  allowing  us  to  build  our  new  research  program  directly  upon  existing 
assets. 

Design  rationale  provided  CDTeam  with  a  current  research  topic  upon  which  to  focus  its  work 
on  collaborative  design.  As  a  result,  the  CDT  Lab  continued  to  offer  the  sort  of  meeting  facilitation 
and  knowledge  elicitation  services  which  local  clients  had  come  to  know  through  the  AKADAM 
project.  This  rehance  on  the  earlier  work  had  several  effects.  By  drawing  on  resident  expertise 
and  tools  derived  from  the  AKADAM  work,  we  were  able  to  minimize  our  ramp-up  costs  and 
expedite  the  transition  to  the  CDT  agenda.  By  sticking  to  knowledge  elicitation  exercises,  we 
obtained  some  measure  of  uniformity  across  the  group  sessions  being  studied  and  a  sound 
foundation  for  evaluating  the  results.  This  approach  even  facilitated  the  recruitment  of  subject 
groups  whose  patterns  of  collaboration  we  could  study.  Due  to  the  popularity  of  the  AKADAM 
work  among  local  clients,  we  had  a  ready  pool  of  potential  subjects  willing  to  conduct  their  real- 
world  design  activities  under  our  scrutiny.  The  contrast  with  AKADAM  lay  in  CDTeam's 
orientation  —  the  services  were  now  being  provided  as  concrete  exercises  within  the  course  of 
which  group  dynamics,  joint  knowledge  representations,  consensus  formation,  and  human 
computer  interaction  could  be  studied. 

As  noted  earlier,  CDTeam  had  concluded  that  general  research  issues  in  collaborative  process 
were  not  unique  to  systems  design,  as  it  is  typically  delineated.  Products  other  than  technical 
artifacts  were  devised  by  processes  of  "design,"  and  we  expanded  the  scope  of  subject  "design" 
activities  to  include  organizational  re-design.  Total  Quahty  Management  (TQM),  Business  Process 
Re-engineering  (BPR),  programmatic  planning,  and  the  creation  of  concept  demonstrations.  By 
providing  facilitation  services  within  CDT  Lab  to  such  broadly-defined  "design"  groups  and 
simultaneously  studying  the  resultant  activities  as  instances  of  collaborative  process,  we  set  the 
stage  for  that  portion  of  our  research  work  termed  simulation  studies.  By  examining  the 
collaborative  design  process  in  its  "natural  setting,"  we  accomplished  multiple  observational 
studies.  By  formulating  and  enacting  formalizations  of  the  key  factors  of  IPD  advantage  (e.g.. 
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design  tradeoffs),  we  were  able  to  pursue  experimental  studies.  Finally,  by  obtaining,  evaluating, 
and  developing  groupware  artifacts,  the  CDT  Lab  was  able  to  pursue  technology  studies.  All  four 
lines  of  research  will  be  discussed  later  in  this  report.  For  now,  we  shall  turn  to  the  second  CDT 
Triangle  "leg"  we  explored  during  this  reported  period  --  Collaborative  Technology. 

Collaborative  Technology 


Introduction:  CSCW  and  Groupware 

As  with  most  information  technology  (IT)  research,  the  areas  most  relevant  to  our  interests  are 
loaded  with  jargon.  The  two  key  labels  for  classifying  our  research  interests  were  Computer- 
Supported  Cooperative  Work  (CSCW)  and  groupware.  "Computer  Supported  Cooperative  Work" 
was  coined  by  Irene  Greif  and  Paul  Cashman  in  1984  as  a  marketing  tag  for  a  vision  of  integrated 
office  IT  support  -  "...A  shorthand  way  of  referring  to  a  set  of  concerns  about  supporting  multiple 
individuals  working  together  with  computer  systems."  (Bannon  &  Schmidt,  1989,  p.  358). 
Generally,  CSCW  pertains  to  the  overall  field  of  supporting  task-oriented  teams  with  information 
technology.  The  term  groupware  is  most  often  invoked  to  reference  those  products  applied  in 
providing  such  support.  This  term  was  first  defined  by  Johnson-Lenz  and  Johnson-Lenz  (1982) 
to  denote  "intentional  GROUP  processes  and  procedures  to  achieve  specific  purposes  plus 
softWARE  tools  designed  to  support  and  facilitate  the  group's  work."  (Hiltz  &  Turoff,  1992).  It 
is  important  in  orienting  oneself  to  emphasize  how  this  initid  definition  subsumes  both  IT  artifacts 
and  the  workplace  social  systems  within  which  they  are  deployed  (Ellis,  Gibbs  &  Rein,  1991). 
Bannon  and  Schmidt  (1989)  discriminate  between  "groupware"  and  group  work  issues,  as  does 
Grudin  (1991).  Following  Johansen  (1988),  Ellis  et  al.  restrict  their  usage  to  the  IT  artifacts, 
themselves,  defining  groupware  as  "computer-based  systems  that  support  groups  of  people 
engaged  in  a  common  task  (or  goal)  and  that  provide  an  interface  to  a  shared  environment"  (1991 , 
p.  40).  For  the  purposes  of  this  report,  we  shall  follow  this  definition. 

Proponents  question  the  precise  boundaries  of  this  research  and  development  area,  though  none 
question  the  value  of  the  issues  addressed  therein.  There  is  a  significant  body  of  research, 
development,  and  trade  literature  covering  the  areas  of  CSCW  and  groupware.  The  reader  wishing 
to  more  deeply  explore  the  origins,  themes,  and  developments  in  this  area  is  recommended  to  Greif 
(1988);  Olson  (1989);  Bostrom,  Watson,  and  Kinney  (1992);  Marca  and  Bt^k  (1992);  and 
Johansen  (1988)  —  all  excellent  introductory  overviews  to  this  area  and  the  issues  it  covers.  For  an 
analytical  overview  of  how  groupware  has  actually  been  applied  in  real  organizations,  see  Bullen 
and  Bennett  (1990a;  1990b).  The  proceedings  from  the  CSCW  conferences  to  date  (Austin  Texas  / 
1986;  Portland  Oregon  /  1988;  London  /  1989;  Los  Angeles  /  1990;  Amsterdam  /  1991;  Toronto  / 
1992;  Milan  /  1993;  Chapel  Hill,  North  Carolina  /  1994)  are  valuable  sources  of  material  at  both 
introductory  and  advanced  levels.  Additional  sources  of  basic  information  are  offered  in 
Appendices  A,  B,  and  C. 

Historical  Background:  Engelbart's  Vision  of  Organizational  IT  Integration 

In  tracing  the  history  behind  groupware,  one  should  start  with  the  work  of  Doug  Engelbart, 
Hating  from  the  1960's.  Engelbart  is  credited  with  many  of  the  innovations  which  now  make 
computers  easier  to  use,  and  he  was  involved  in  the  creation  of  the  earliest  computer-supported 
meeting  environment  at  the  Stanford  Research  Institute  in  the  1960's.  It  is  Engelbart's  overall 
vision  of  how  computers  can  be  employed  in  organizations  which  both  sets  the  context  for  these 
individual  achievements  and  establishes  him  as  a  key  source  of  the  ideas  in  CSCW.  Engelbart's 
vision  was  one  in  which  knowledge  workers  deal  with  information  rather  than  with  physical  goods 
(1982).  In  addition  to  manipulating  and  manufacturing  data,  they  create  knowledge  of  the  task,  of 
the  means  for  achieving  that  task,  and  of  the  workplace.  Shared  information  environments  provide 
the  milieux  within  which  knowledge  workers  can  augment  as  well  as  mutually  pool  knowledge. 
Some  key  features  in  Engelbart's  vision  are: 
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•  access  to  computers  for  all  workers  (including  easy  usability); 

•  linkages  among  all  workers  within  an  organization  via  telecommunications; 

•  storage  of  the  organization's  "knowledge"  within  this  shared  electronic 
environment;  and 

•  the  means  by  which  the  ongoing  "knowledge"  relating  to  operations  can  accrete  to 
the  shared  environment. 

Engelbart  himself  has  pursued  this  vision  over  the  last  4  decades.  A  prototype  system  (called 
Augment)  was  developed,  incorporating  many  of  the  communications  /  storage  /  retrieval  / 
documentation  functions  which  today  we  associate  with  email,  hypertext,  databases,  and  the  like. 
Augment  later  served  as  the  foundation  for  data  services  provided  through  Tymeshare,  Inc.,  which 
even  later  was  bought  by  McDonnell  Douglas.  Today,  Augment  is  a  component  of  the  tools  being 
applied  by  Engelbart's  Bootstrap  Institute  in  his  ongoing  efforts  to  investigate  the  ways  in  which 
organizations  can  implement  shared  knowledge  environments  and  apply  the  results  toward  renewal 
of  large  American  institutions  (particularly  corporations)  (Engelbart,  1990). 

What  Delineates  CSCW  and  Groupware? 

Some  authors  have  given  their  attention  to  discussing  work,  while  others  concentrate  on 
products,  and  this  divergence  of  focus  (i.e.,  products  versus  work  processes)  has  been  widely 
recognized  and  discussed.  Bannon  and  Schmidt  (1989,  p.  359)  distinguish  between  "CS"  and 
"CW"  in  illustrating  this  divide,  while  Grudin  (1991)  contrasts  "groupware"  with  "CSCW." 
Whitaker,  Ostberg,  and  Essler  (1989)  distinguish  between  CScw  {Computer  Support  for 
cooperative  work  —  the  technical  perspective)  and  csCW  (computer  support  for  Cooperative  Work 
-  the  social  /  organizational  perspective).  This  version  of  the  distinction  will  be  employed  in  this 
document.  A  closely  related  distinction  is  geographically  determined  between  the  product-oriented, 
CScw  tenor  of  American  work  and  the  organizational-oriented,  csCW  emphasis  more  common  in 
Europe  (Grudin,  1991). 

Largely  due  to  these  differential  foci,  CSCW  is  impossible  (and  groupware  very  difficult  ...)  to 
precisely  circumscribe  (cf.  Howard,  1987;  Bannon  &  Schmidt,  1989;  Grudin,  1991);  and 
Robinson,  1989;  1991).  The  central  notions  of  computer  support  and  cooperative  work  provide  a 
basic  agenda  for  discussion.  Problems  arise,  however,  when  one  moves  "outward"  from  this 
central  area  and  meets  with  other  disciplines  and  research  areas.  Some  general  contrasts  can  be 
drawn  so  as  to  generally  orient  our  discussion  by  delimiting  the  "boundaries"  circumscribing 
CSCW  and  groupware. 

Groupware  is  not  synonymous  with  any  class  of  IT  product.  Groupware  is  explicitly  designed 
to  support  collective  activity  among  workers,  so  it  is  defined  partially  by  the  collective  nature  of  the 
work  it  supports.  Even  this  qualification  fails  to  precisely  delineate  the  field,  because  the  variety  of 
support  systems  and  strategies  is  as  great  as  the  variety  of  activities  and  interactions  in  which 
people  collectively  engage  (Johansen,  1989a).  As  a  result,  many  diverse  artifacts  are  called 
groupware,  with  different  authors'  categories  differentially  denoting  variants  and  composites 
within  the  potpourri.  This  variety  weakens  the  label's  utility  in  categorizing  software  applications 
(Opper,  1988).  Conversely,  this  profligate  variety  confounds  attempts  to  define  CSCW  via 
enumeration  of  products  (cf.  Wilson,  1988). 

Neither  is  groupware  completely  specified  by  its  IT  implementation  environment.  Grudin 
(1991,  p.  6)  outlines  unresolved  contradictory  views  on  delineating  "groupware"  vis  a  vis 
foundational  technologies.  Engelbart's  original  support  tools  for  "knowledge  workers"  (1963; 
1982)  —  the  harbingers  of  CSCW  —  were  conceived  and  implemented  in  the  days  of  mainframes 
and  dumb  terminals.  The  strong  association  of  groupware  products  with  multi-user  environments 
should  not  be  considered  an  equivalence  relation.  In  a  multi-user  environment,  many  people  work 
simultaneously,  but  there  is  no  connotation  their  tasks  are  interdependent  (Whitaker  &  Essler, 


1990;  F.llis  et  ai,  1991).  Indeed,  undue  reliance  upon  analogies  drawn  from  earlier  multi-user 
systems  has  been  identified  as  one  critical  factor  in  groupware  failures  (Grudin,  1988). 

Ellis,  Gibbs,  and  Rein  (1991)  conclude  it  is  most  reasonable  to  think  in  terms  of  a  spectmm 
within  which  applications  can  fall  as  "groupware."  They  define  this  "groupware  spec^m"  in 
terms  of  (1)  degree  of  commonality  in  task  and  (2)  degree  of  commonality  in  the  (electronic)  work 
environment  —  thus  spanning  both  approaches  in  one  loose  composite  measure.  In  other  words, 
specific  systems  or  environments  provide  only  a  context  in  which  groups  may  collaborate; 
collaborative  activity  itself  is  the  necessary  condition  for  the  label  "groupware.  ” 

Groupware  is  not  synonymous  with  communications.  Because  groupware  implies 
collaboration,  its  delineation  and  implementation  are  necessarily  intertwined  with  issues  of 
communication.  This  is  reflected  in  the  typical  targeting  of  groupware  products  for  LAN 
environments  (Johansen,  1989)  and  telecommunications  companies'  high  profile  in  CSCW 
hterature  and  the  CSCW  conferences  to  date  (Grudin,  1991).  At  the  extreme,  Wright  (1990) 
addresses  CSCW  (as  distributed  group  work  —  i.e.,  different  time  and/or  place)  as  one  of  several 
emerging  trends  within  telecommunications  itself  Telecommunications,  however,  is  not 
equivalent  to  CSCW  or  groupware.  Computer-supported  meeting  rooms  are  considered 
groupware  (cf  Johansen,  1989b),  but  much  of  the  supported  communication  is  enacted  in  the 
ambient  social  space,  not  through  the  artifacts  themselves.  Johansen  (1989b)  lists  many  general 
purpose  communication  systems  in  his  groupware  review,  but  does  so  within  the  broad  context  of 
"technological  support  for  work  group  collaboration,"  basing  their  inclusion  on  provision  of  a 
medium  for  collective  activities. 

These  exceptions  reveal  that  access  to  a  common  medium  within  the  context  of  collective 
activity,  rather  than  communication  per  se,  is  the  fundament  for  CSCW.  This  "shared  information 
space"  concept  is  implicit  in  Engelbart  (cf  1963;  1982)  and  (termed  shared  environment)  is  an 
integral  part  of  Ellis  et  al.'s  (1991,  p.  40)  definition  for  "groupware."  Both  Robinson  (1991)  and 
Bannon  (1991b)  allude  to  the  work  of  Thompson  (1984)  with  regard  to  this  concept,  but  do  not 
make  any  claim  that  Thompson  is  the  origin  of  the  phrase  in  the  sense  that  it  has  become  an 
important  "CSCW  specific  concept"  (Robinson,  1991).  Bannon  and  Schmidt  (1989,  p.  364) 
identify  "sharing  an  information  space"  as  a  "core  issue  for  CSCW";  and  and  De  Michelis  (Butler 
Cox  Foundation,  1990)  cites  "information  sharing"  as  the  key  support  for  collaborative  activity.  In 
all  these  cases  the  focus  is  on  common  access  to  task-specific  information  rather  than  on  the 
communication  links  via  which  that  access  is  realized.  This  distinction  between  the  technical 
foundation  (communications  infrastructure)  and  the  operational  benefit  (shared  information  space) 
is  addressed  in  Bannon  (1989)  and  in  Grudin's  comments  (1991)  on  databases  and  CSCW.  In 
other  words,  communications  systems  are  groupware  only  to  the  extent  they  specifically  lend 
support  to  some  collaborative  activity. 

CSCW  is  not  defined  solely  in  terms  of  cooperation.  The  term  CSCW  becomes  no  clearer 
when  approached  from  the  direction  of  work  process.  After  an  extensive  review,  Bannon  and 
Schmidt  (1989)  conclude  "...the  term  'cooperative  work'  is  the  general  and  neutral  designation  of 
multiple  persons  working  together  to  produce  a  product  or  service"  (p.  362).  Their  lack  of  further 
specificity  is  understandable,  because  "cooperation"  becomes  a  very  problematic  subject  under 
closer  scrutiny.  Those  who  have  gone  this  route  —  retaining  a  broader  social  scope  and  addressing 
general  interpersonal  factors  -  have  ended  up  at  the  extreme  of  framing  "cooperation"  as  a  purely 
sociopolitical  phenomenon. 

Howard  (1987,  p.  175-176)  criticizes  such  extreme  views  of  cooperation  as  being  "...not 
merely  a  description  of  the  way  work  is  but  a  prescription  for  the  way  it  ought  to  be."  Bannon  and 
Schmidt  (1989)  close  off  this  line  of  definition  when  they  state  "(t)he  concept  of  cooperative  work 
does  not  imply  a  particular  degree  of  participation  or  self-determination  on  the  part  of  the  workers, 
nor  a  particularly  democratic  management  style"  (p.  362).  This  assertion  is  substantiated  by 


experiences  in  groupware  implementation.  Caracik  and  Grantham  (1988)  describe  users’  rejection 
of  The  Coordinator™  with  reference  to  its  perceived  negative  impact  on  autonomy  and  equality; 
Bullen  and  Bennett  (1990a;  1990b)  note  the  social  impacts  of  groupware  implementations;  and 
Whitaker,  Ostberg,  and  Essler  (1989)  suggest  that  groupware  products’  intrinsic  presumptions 
may  occasionally  violate  mores  of  national  as  well  as  corporate  cultures. 

Like  the  earlier  points  discussed,  consideration  of  team  work  as  the  sole  discriminant  in  CSCW 
leads  to  ambiguity.  If  a  CSCW  application  is  defined  by  work  setting,  and  "all  human  activity  is  in 
some  sense  ’cooperative’  "  (Howard,  1987,  p.  175),  it  is  difficult  to  see  how  any  system  could 
avoid  being  so  categorized.  If  one  speaks  broadly  enough  about  tlie  "task"  of  an  entire 
organization  (corporation,  agency,  etc.),  then  one  can  subsume  all  workers  within  a  group  whose 
goal  is  achievement  of  this  task,  diluting  the  idea  to  near-uselessness.  In  other  words,  the  activity 
of  interest  is  defined  jointly  in  terms  of  interacting  collaborators  and  shared  goal(s),  not  in  terms  of 
any  a  priori  quality  defined  socially  or  politically. 

CSCW  is  not  delineated  with  strict  regard  to  organizational  boundaries.  Skeptics  have  long 
viewed  CSCW  as  a  repetition  of  the  q^/ce  automation  fad  of  the  late  1970’s,  emphasizing 
integrated  production  support  for  entire  organizational  units  (e.g.,  Wohl,  1989).  With  specific 
regard  to  strict  organizational  delineation,  such  comparisons  are  unfounded.  Grudin  (1988) 
illustrates  populations  affected  by  CSCW  implementations  do  not  necessarily  correspond  to  the 
entirety  of  an  organization.  Conversely,  consideration  of  CSCW  need  not  be  properly  constrained 
to  a  single  organization.  Toffler’s  (1990)  interconnected  "power  mosaics"  (flexible  collaborative 
networks,  successors  to  monolithic  enterprises)  rely  on  inter-organizational  communication  and 
coordination,  and  they  will  provide  a  major  impetus  to  groupw'are  proliferation.  Suomi  (1989)  and 
Hart  and  Estrin  (1990)  provide  general  overviews  of  IT  systems  for  coordinating  operations  across 
organizational  boundaries,  while  Engelbart  (1990)  describes  an  example  of  "knowledge  domain 
interoperability"  from  the  aerospace  industry,  spanning  some  6,000  separate  companies. 

Groupware  cannot,  therefore,  be  precisely  mapped  onto  organizational  units.  This  is  just  as 
well  —  if  we  accept  Toffler’s  (1990)  vision  of  emerging  "flex-firms"  and  the  "power  mosaics," 
then  (respectively)  internal  architectures  and  rigid  inter-organizational  relations  are  becoming 
obsolete  as  useful  delimiters.  What,  then,  is  a  more  appropriate  scope  for  consideration?  Grudin 
(1988)  suggests  addressing  the  aforementioned  conflicts  by  restricting  consideration  to  "...smaller 
or  more  homogenous  groups,"  claiming  "...there  may  be  less  bias  when  only  peer-peer 
communication  is  involved  than  when  the  communication  moves  vertically  through  the 
organizational  hierarchy"  (p.  87).  Markus  and  Connolly  (1990)  extend  Grudin’s  discussion  by 
reference  to  interdependence  among  users  of  specific  applications,  rather  than  classes  or  subunits 
defined  with  respect  to  the  organizational  map.  In  both  cases,  discussion  occurs  against  a 
backdrop  of  organizational  structure,  but  andysis  defaults  to  task-oriented  relations  among 
interactors.  In  other  v/ords,  the  relations  delimiting  the  groups  of  interest  are  relations  of  concert^ 
activity,  not  relations  of  organizational  membership  or  ranking. 

C5CW  is  not  isomorphic  with  other  disciplines  or  research  fields .  A  certain  interdisciplinary 
flexibility  is  implied  (and  demanded)  in  addressing  the  confluence  of  IT  and  group  activities. 
Grudin  (1991)  identifies  the  MIS  and  HCI  communities  as  "...the  major  contributors  to  groupware 
development  and  CSCW  research"  (p.  12).  Ellis,  Gibbs,  and  Rein  (1991)  list  "distributed 
systems,  communications,  human-computer  interaction,  artificial  intelligence  (AI),  and  social 
theory"  as  "five  key  disciplines  or  perspectives  for  successful  groupware"  (p.  44).  We  must  take 
care  to  distinguish  the  perspectives  of  research  fields  intersecting  CSCW  from  that  perspective 
which  (however  indistinctly)  identifies  CSCW  itself  This  is  critical,  owing  to  (1)  the  diversity  of 
venues  in  which  CSCW  research  is  presented  as  well  as  (2)  the  diversity  of  research  presented  in 
venues  ostensibly  centered  on  CSCW. 

An  example  of  the  first  case  concerns  the  human-computer  interaction  (HCI)  /  human  factors 


(HF)  community.  CSCW  has  been  a  persistent  topic  at  the  ACM  CHI  conferences,  and  ACM's 
SIGCHI  co-sponsors  the  North  American  CSCW  conferences.  However,  the  ability  of  HF/HCI 
research  to  contribute  to  CSCW  is  limited.  The  HF  community  has  tended  to  address  "knowledge 
workers"  in  terms  recycled  from  the  days  of  mechanical  automation  ~  functionality,  efficiency,  and 
the  impacts  of  technology  on  individual  workers.  HCI's  attention  to  the  interface  between 
computers  and  human  "users"  narrowly  addresses  an  artifact's  functionality  with  respect  to  an 
individual's  physical  and  cognitive  capacities  (Grudin,  1990a;  1990b).  Historically,  this  viewpoint 
derives  from  HF  research's  original  industrial  setting,  and  it  is  perpetuated  by  HCI's  reliance  on 
laboratory  experimentation,  isolating  system  usage  from  its  workaday  context  (Bannon,  1991a). 

This  results  not  only  in  impractical  abstraction  and  rapid  obsolescence,  but  (more  importantly) 
a  blindness  to  group  support  issues  such  as  interaction  (Bannon,  1991a).  This  blindness  is  not 
overcome  through  simple  analogies  between  functional  units.  Work  teams  are  flexible,  dynamic, 
often  transient  social  networks  presumably  best  analyzed  via  social  science  techniques.  HF/HCI 
(as  a  scientific/engineering  enterprise)  leaps  from  the  level  of  the  individual  to  that  of  a  composite, 
indivisible  unit  (company,  union,  etc.).  The  complexities  of  work  group  interactivity  cannot  be 
addressed  via  extrapolations  from  the  individual  user  to  the  organization  itself  or  extrapolations 
from  individual  users  to  entire  categories  or  classes  of  workers. 

An  example  of  the  second  case  concerns  Scandinavian  participatory  design  (PD).  The  strong 
association  of  PD  with  CSCW  is  understandable  to  the  extent  that:  (1)  both  fields  emphasize  the 
social  and  organizational  aspects  of  working  life  and  (2)  collaborative  design  exercises  are 
instances  of  "cooperative  work"  (Whitaker,  Essler,  &  Ostberg,  1991).  On  the  other  hand,  the 
workplace  collaboration  emphasized  in  CSCW  is  not  synonymous  with  the  workplace  democracy 
emphasized  in  PD,  and  there  is  a  big  difference  between  collaborating  on  an  IT  design  and 
designing  IT  for  collaboration.  PD  is  often  pursued  with  groups  of  workers,  but  there  is  no 
presumption  the  system  being  designed  is  groupware.  Conversely,  much  of  the  design  activity 
characterized  by  participatory  design  writers  as  "cooperative"  is  accomplished  without  any  direct  IT 
support. 

There  are  as  many  points  of  divergence  as  of  correspondence  between  CSCW  and  the  other 
fields  to  which  it  exports  and  from  which  it  imports  reported  research.  HCI  has  informed  us  on 
some  aspects  of  CScw,  but  it  is  conceptually  and  methodologically  ill-equipped  to  address  csCW 
(cf.  Bannon,  1991a).  PD  has  addressed  csCW,  but  is  not  intrinsically  linked  to  groupware  -- 
either  in  terms  of  its  tools  or  its  products  (Whitaker,  Essler,  &  Ostberg,  1991).  In  other  words, 
CSCW  is  interdisciplinary  and  irreducible  to  either  CScw  or  csCW.  Work  ascribed  to  CSCW 
should  be  assessed  in  light  of  the  "perspective"  from  which  it  is  undertaken.  Work  imported  from 
other  research  areas  should  be  qualified  with  respect  to  their  native  "perspectives.  ” 

CSCW  Defined  bv  Task  Environment:  The  Example  of  Johansen 

Robert  Johansen  (1989b)  provides  a  taxonomy  of  groupware  applications  using  distribution  in 
time  and  space  to  delimit  application  categories,  as  partially  replicated  in  Table  1.  He  categorizes 
some  of  the  17  different  types  of  group  support  applications  he  identifies  (Johansen,  1988;  1989a) 
into  subsets  based  on  the  dispersion  or  copresence  of  collaborating  parties  in  time  and/or  space. 
This  categorization  scheme  has  proven  particularly  useful  both  as  a  means  of  classifying 
groupware  products  and  as  an  illustrative  device  for  demonstrating  the  types  of  work  environments 
addressable  with  such  products.  Johansen  et  al.  (1991)  claim  Ae  inspiration  for  the  time/space 
classification  came  from  DeSanctis  and  Gallupe's  (1987)  discussion  of  GDSS.  One  advantage  of 
Johansen's  approach  is  that  newcomers  to  the  notion  of  CSCW  can  easily  grasp  the  time  /  space 
permutations  and  his  mapping  of  product  classes  onto  them.  Also,  by  making  time  and  space  the 
key  dimensions  for  his  matrix,  he  has  managed  to  avoid  the  thorny  issues  of  what  one  means  by 
collaborative  work  and  the  non-informative  nature  of  a  simple  listing  of  products.  The  result  is  a 
perspective  which  is  useful  without  either  forcing  a  priori  decisions  on  ill-defined  or  vague 


characteristics  (e.g.,  mutuality  of  goals,  reciprocal  benevolence)  or  limiting  oneself  to  any 
collection  of  specific  market  entities. 


Table  1  CSCW  Applications  Based  on  Distribution  in  Time  and  Space  (Adapted  from  Johansen, 
1989b) 


SAME  TIME  DIFFERENT  TIMES 


FACE-TO-FACE 

/ADMINISTRATION/ 

MEETINGS 

DATA  MANAGEMENT 

Copyboards 

Shared  files 

SAME  PLACE 

PC  projectors 

Meeting  rooms 

Shift  work 

REMOTE 

RELIANCE  ON 

MEETINGS 

COORDINATION 

Conference  calls 

Electronic  mail 

DIFFERENT 

Data  sharing 

Forms  management 

PLACES 

Video/Tele-conferencing 

Voice  mail 

Structured  messaging 

CSCW  Defined  via  Functionality:  The  Example  of  De  Michelis 

A  more  recent  discussion  of  CSCW  —  De  Michelis  (1990)  —  differs  from  the  CScw  and  csCW 
viewpoints  in  attempting  to  categorize  cooperative  work  processes  to  provide  a  means  for 
discussing  specific  software  applications.  His  approach  concentrates  on  the  mode  of  cooperative 
activity,  rather  than  on  a  comprehensive  definition  of  cooperation  itself.  De  Michelis  does  not 
attempt  to  define  what  he  means  by  "cooperation."  Instead,  he  notes  a  trend  toward  the  use  of 
task-directed  groups  in  modem  enterprises  and  claims  those  groups  are  "...defined  by  the  pattern 
of  commitments  that  group  members  make  with  each  other  and  with  third  parties"  (p.  2). 

Having  taken  the  group  as  his  focus,  De  Michelis  proceeds  to  delineate  three  different 
categories  of  cooperation:  coordination,  collaboration,  and  co-decision.  Coordination  is  that 
process  by  which  group  members  organize  and/or  synchronize  their  actions  within  the  framework 
of  a  task.  Collaboration  consists  of  those  activities  through  which  multiple  actors  work  together  on 
a  given  task.  Co-decision  is  an  extended  form  of  collaboration  in  wluch  the  task  is  reaching  a 
decision.  Based  on  this  trinary  distinction,  De  Michelis  then  proceeds  to  discuss  specific  types  of 
support  systems  developed  to  date.  He  does  not  address  a  general  process  of  "cooperation,"  so  it 
is  difficult  to  assign  his  analysis  to  the  csCW  camp.  Because  he  is  proceeding  from  a  basis  of 
work  style,  it  is  difficult  to  see  him  as  purely  in  the  CScw  vein. 

If  one  looks  carefully  at  De  Michelis'  classifications,  the  boundaries  among  them  immediately 
blur,  if  not  disappear  entirely.  De  Michelis  himself  implies  that  co-decision  is  a  variant  on 
collaboration.  Coordination  can  be  re-interpreted  as  either  (1)  a  form  of  collaboration  within  which 
the  goal  is  resource  allocation  and/or  synchronization;  or  (2)  a  form  of  co-decision  with  the  same 
focus.  Clearly,  this  trifold  framework  cannot  be  maintained  as  a  general  analytical  tool,  although  it 
has  merit  as  an  illustrative  device.  More  important  than  his  absolute  accuracy  is  De  Michelis'  shift 
of  definitional  emphasis  from  a  general  notion  of  "cooperative  work"  to  more  specific,  functionally 
delineable  classes  of  activities.  Whether  or  not  one  accepts  his  view  of  coordination,  collaboration. 


and  co-decision  as  fundamental  categories,  they  provide  a  useful  means  for  addressing  the  types  of 
activities  subsumed  in  CSCW  without  falling  prey  to  the  ambiguities  and  subjective  values  which 
have  plagued  attempts  to  define  "cooperative  work"  generally.  Siniilar  to  Johansen,  who  improved 
the  "focus"  of  product  enumerations,  De  Michelis  enhances  the  clarity  with  which  the  activities  are 
addressable. 

Johansen  accomplished  his  clarification  by  adding  the  referential  dimensions  of  time  and  space. 
De  Michelis  makes  similar  progress  by  adding  discriimnatory  criteria  of  specific  work  goals.  'Die 
goal  of  coordination  concerns  the  plans  for  accomplishing  a  given  task;  the  goal  of  collaboration 
concerns  the  actions  by  which  that  task  is  accomplished;  and  the  goal  of  co-decision  concerns 
policies  with  regard  to  some  task  or  topic.  These  descriptions'  relative  concreteness  derives  from 
their  being  phrased  in  terms  of  goals  or  results.  De  Michelis'  primary  contribution  is  therefore  the 
addition  of  these  goal-directed  criteria,  which  enables  him  to  address  activities  more  precisely  than 
earlier  CSCW  analysts. 

CSCW  Defined  bv  the  Existence  of  Groups 

Early  attempts  to  define  CSCW  as  an  area  of  interest  have  fallen  either  into  the  CScw  or  the 
csCW  foci.  The  writers  who  have  managed  to  somehow  surmount  the  problems  of  this  dichotomy 
-  Johansen  and  De  Michelis  -  have  done  so  by  shifting  focus.  In  the  case  of  Johansen  (who 
refined  the  CScw  angle),  the  focus  is  moved  to  a  matrix  of  time  and  space  parameters.  In  the  case 
of  De  Michelis  (who  refined  the  csCW  angle),  the  focus  is  moved  to  variations  among  task  goals 
and/or  results.  In  both  cases,  the  dichotomy  is  circumvented  by  ceasing  to  see  CSCW  (and  CSCW 
products)  as  being  defined  via  some  vaguely  defined  "collective  activity"  and  instead  coming  to 
view  whatever  is  identifiable  as  collective  activity  as  being  defined  with  respect  to  something  more 
tangible  -  an  identifiable  group  who  (among  other  things)  share  a  task  and  the  means  for 
accomplishing  that  task. 

In  the  case  of  Johansen,  this  is  illustrated  by  reference  to  parameter  wWch  map  relations 
among  people  (not  products)  who  are  distributed  in  time  and  space.  De  Michelis  (1990)  explicitly 
takes  the  stance  that  the  fundamental  distinction  involves  collections  of  individuals  rather  than  some 
particular  variety  of  work,  stating  whatever  activities  we  categorize  as  some  sort  of  pooperative 
work  are  known  to  be  thus  categorizable  by  reference  to  discernible  groups  and  the  "...different 
types  of  objectives,  communication,  and  relationships..."  (p.  2)  holding  among  their  members.  In 
other  words,  one  key  criterion  for  discriminating  this  ephemeral  CSCW  area  is  the  group  itself  — 
neither  some  feature  or  quality  of  its  activities  nor  some  specific  character  of  the  tool(s)  it  employs 
(Whitaker  &  Essler,  1990).  Dimensions  or  features  pertinent  to  this  criterion  are: 


•  Identity  of  the  set  of  specific  individuals  which  delineate  the  group  as  a  collection  of 

people; 

•  Identity  of  the  network(s)  of  interactivity  among  this  set  of  individuals; 

•  Identity  of  the  roles  or  functions  characterizing  each  individual's  participation  in  the 

group;  ,  .  ,  . 

•  Identity  of  the  task  or  goal  which  delineates  the  group  as  a  functional  unit. 

This  emphasis  on  the  group  (or  workplace  social  system)  is  not  offered  as  an  exclusive 
alternative  to  either  time/space  parameters  or  goal  specifications;  indeed,  all  three  of  these  aspects 
mutually  influence  each  other.  One  can  offer  examples  of  groups  which  do  not  operate  in  a 
collaborative  fashion  (e.g.,  the  set  of  all  users  in  a  centralized  multi-user  system),  and  one  may 
point  to  software  tools  designed  for  individual  users  which  are  nonetheless  employed  for 
collaborative  ends  (word  processors  used  during  small  group  brainstorming  sessions).  However, 
there  is  no  such  thing  as  collaboration  without  there  being  a  group  thereby  co-defined  -  delineation 
of  a  "group"  is  therefore  of  particular  importance  in  delineating  "group  work."  Taken  together. 


characteristics  of  the  work  environment,  the  work  results,  and  the  work  group  jointly  define  those 
workspaces  which  have  heretofore  been  addressed  by  CSCW  researchers.  Furthermore,  they 
accomplish  this  through  reference  to  specifiable  elements  (e.g.,  time,  space,  goals,  results,  actors, 
and  patterns  of  interactivity). 

The  shortcomings  of  CSCW  characterizations  critiqued  earlier  derived  from  assuming  the 
artifacts  or  work  activity  exhibiting  specific  styles  (e.g.,  mutuality  of  goals;  equality  of 
empowennent)  were  determinant.  Johansen’s  and  De  Michelis'  critical  improvements  concern 
criteria  which  are  themselves  defined  relative  to  participants  in  a  given  collective  activity.  Time  and 
space  intervene  between  workers,  not  just  their  tools.  Goals  and  results  are  determined,  attempted, 
and  evaluated  by  workers,  not  some  objective  guarantor.  In  other  words,  groupware  is  identifiable 
only  with  respect  to  groups,  and  group  work  is  identifiable  only  with  respect  to  interactivity. 
Based  on  this  analysis,  we  felt  confident  we  could  proceed  with  CSCW-relevant  research  framed 
in  terms  of  concrete  project  teams,  circumstances,  and  parameters. 

Summary 


CSCW  is  not  comprehensively  characterized  either  by  artifacts  or  activities.  Analysis  from  the 
basis  of  stereotyped  group  activities  (i.e.,  csCW)  leads  to  prescriptions  for  reconciling 
sociopoliticd  factors  (e.g.,  empoweraient)  when  intervening  into  workplaces  portrayed  as 
microsocieties  (e.g.,  PD).  This  orientation  is  not  necessarily  informative  on  technological 
interventions.  Whitaker,  Essler,  and  Ostberg  (1991)  suggest  that  Scandinavian  PD's  tme  subject 
matter  is  organizational  redesign  rather  than  the  design  of  IT  artifacts,  based  on  case  examples 
where  participatory  practices  were  applied  to  model  the  workers'  organization  in  preparation  for  IT 
development.  Analysis  from  the  basis  of  work  tools  (i.e.,  CScw)  leads  to  prescriptions  for 
reconciling  human  factors  (e.g.,  performance  metrics)  when  intervening  into  workplaces  portrayed 
as  functional  units  (e.g.,  HF/HCI).  This  orientation  is  not  necessarily  informative  on  social 
interventions.  This  is  not  to  say  the  specific  fields  mentioned  (PD  and  HF/HCI)  are  of  no  use  to 
CSCW  researchers.  The  reciprocal  influences  of  social  and  technical  factors  require  that  analysis 
and  intervention  be  sensitive  to  both  sides.  The  complementary  blindnesses  of  (extreme)  CScw 
and  csCW  require  that  progress  will  entail  explanations  spanning  individual  actors  and  groups  of 
interactors. 

Consideration  of  groupware  implementation  should  begin  with  identification  and  evaluation  of 
the  work  team  itself.  The  characteristics  of  the  work  environment  and  the  work  gtoup  jointly 
define  those  workspaces  which  are  amenable  to  groupware  solutions.  The  specifiable  elements 
emphasized  in  useful  characterizations  of  CSCW  settings  (e.g.,  time,  space,  goals,  results,  actors, 
and  patterns  of  interactivity)  all  pertain  to  the  team,  its  members,  and  the  patterns  of  interactivity 
among  them.  Taken  in  conjunction  with  the  preceding  conclusions,  this  means  we  should  explore 
workplaces  neither  as  unitary  microsocieties  nor  as  collections  of  information  processing  units 
The  common  link  between  the  csCW  (e.g.,  PD)  and  the  CScw  (e.g.,  HF/HCI)  "perspectives"  is 
the  indiyidu^  human.  However,  in  the  former  persfjective  the  individual  is  often  subsumed  within 
a  bloc  (i.e.,  linked  into  the  workplace  social  system  only  in  terms  of  a  depersonalized  role),  while 
in  the  latter  the  individual  is  often  regarded  only  in  the  context  of  direct  engagement  with  an  IT 
system  (i.e.,  not  in  the  context  of  the  workplace  social  system).  Phrased  another  way,  the  csCW 
perspective  sometimes  abstracts  the  worker  as  a  unit  of  socio-political  activity,  while  the  CScw 
perspective  is  evident  in  HCI/HF  work  which  stereotypes  workers  in  terms  of  "user  models." 


CDT  RESEARCH  INFRASTRUCTURE 
CDT  Lab:  A  Reconfigurable  Group  Testbed 

In  September  1993  the  CDT  Lab  began  its  "ramp-up"  process  by  acquiring  a  large  laboratory 
space  and  requisitioning  equipment  to  set  up  an  advanced  technology  suite.  Our  plan  was  to 
construct  a  modular,  flexibly-configurable  testbed  providing  (with  IT  support)  a  unified  milieu  for 
group  activities  and  a  simulation  platform  for  both  co-located  and  remotely  distributed  team  tasks. 
Because  we  planned  to  follow  the  example  of  AKADAM  and  exploit  opportunities  for  evaluating 
collaborative  technologies  in  the  context  of  real-world  tasks  (e.g.,  USAF  design  projects),  we  had 
to  ensure  that  the  CDT  Lab  was  equipped  to  handle  such  tasks’  demands.  This  need  for  concrete 
utility  in  workaday  operations  further  constrained  us  to  equip  the  CDT  Lab  with  some  degree  of 
conformance  or  compatibility  with  the  IT  support  already  employed  in  those  operations. 

As  discussed  in  the  introduction  to  CSCW  and  groupware,  this  area  of  research  is  technology¬ 
intensive.  Even  for  the  simplest  case  of  equipping  a  meeting  room  (e.g.,  the  AL/CFH  Multimedia 
Room)  for  electronic  display  and  software  support,  a  ^tentially  high  investment  in  hardware  and 
software  is  unavoidable.  Affording  collaborators  individual  access  and  control  to  electronic  media 
multiplies  this  unavoidable  investment  in  a  roughly  geometrical  fashion.  The  CDT  Lab  mission 
statement's  emphasis  on  the  “distributed”  aspect  further  increased  (1)  the  reliance  on  technology  in 
setting  our  research  context  and  (2)  the  explicit  and  implicit  costs  for  our  ongoing  work.  The 
increase  in  reliance  resulted  from  the  IT  infrastructure’s  role  as  communication  medium  as  well  as 
task  tool.  The  increase  in  costs  resulted  from  the  overhead  needed  to  provide  communications 
channels  overcoming  collaborators’  separation  in  time  and  /  or  space. 

Because  our  research  infrastructure  needed  to  provide  for  both  the  co-located  ^d  distributed 
scenarios,  we  faced  the  prospect  of  very  high  ramp-up  costs.  This  was  primarily  due  to  the 
inherent  resource  redundancies  of  (1)  providing  similar  support  to  multiple  collaborators  within  a 
team  (e.g.,  computer  workstations  and  groupware)  and/or  (2)  providing  for  similar  team  scenarios 
in  distributed  settings  (e.g.,  room-to-room  video  tel^onferencing).  Given  our  finite  budgetary 
means,  we  therefore  devised  our  initial  laboratory  architecture  in  such  a  way  as  to: 

•  Maximize  the  utility  of  CDT  Lab  assets  across  the  range  of  time  /  space  scenarios. 

•  Maximize  the  uniformity  of  overall  IT  assets  (especially  the  computers)  within  CDT 

Lab  itself. 

•  Maximize  the  compatibility  of  CDT  IT  assets  with  those  already  employed  by  potential 

USAF  clients. 

•  Maximize  the  utilization  of  existing  AL/CFHD  assets  as  auxiliary  resources  for  CDT 

Lab. 

The  following  sections  will  outline  background  issues  and  our  planning  with  regard  to 
physical,  computing,  and  communication  resources. 

Physical  Factors  in  Group  Support 

The  physical  meeting  environment  (e.g.,  a  computer-supported  conference  rooni)  has  long 
been  recognized  as  a  problematic  component  of  the  group  meeting  support  "package."  Physical 
(architectural,  lighting,  seating)  factors  have  been  the  subject  of  much  research  (e.g.,  Ferwanger, 
etai,  1989;  Mantei,  1988;  Olson  et  al.,  1990),  because  the  character  and  affordances  of  the  room 
setting  influence  the  decision  making  process.  Such  influence  is  typically  nianifested  through 
constraints  on  representational  support  and  constraints  on  the  available  modalities  for  dialogue. 
For  example,  Ferwanger  et  al.  (1989)  cite  several  ways  physical  arrangement  of  a  meeting  site  can 
influence  interaction  either  through  direct  functional  biases  or  tacit  social  cues.  The  types  of  design 
tradeoffs  to  be  considered  in  computer-supported  meeting  room  design  are  well-illustrated  by  two 


similar  facilities  in  Ann  Arbor,  Michigan  —  the  EDS  Capture  Lab  and  the  University  of  Michigan's 
Collaboration  Technology  Suite  within  the  Cognitive  Science  and  Machine  Intelligence  Laboratory 
(CSMIL). 

An  example  of  the  attention  physical  layout  requires  is  the  evolution  of  the  central  conference 
table  at  these  two  facilities.  At  EDS  Capture  Lab,  participants  originally  had  vertical  computer 
displays,  angled  toward  the  common  wdl  display  at  the  end  of  the  table.  This  was  meant  to 
minimize  the  distance  between  the  individual  and  group  displays,  so  that  one  need  only  look  up 
from  his/her  CRT  to  see  the  wall  display.  This  well-intentioned  layout  proved  disadvantageous  in 
use.  First,  the  vertical  individual  displays  were  partial  obstacles  to  direct  face-to-face  interactions 
among  participants.  Second,  the  viewing  of  both  displays  in  such  close  (angular)  proximity 
seemed  to  result  in  a  lack  of  "visual  privacy"  for  each  individual  display.  Third,  the  architecturally- 
enforced  orientation  to  the  common  wall  display  conflicted  with  the  participants'  need  to  orient  to 
each  other  during  direct  interaction.  Finally,  participants  reported  physical  stress  and  discomfort 
from  the  angled  arrangement  (Mantei,  1988). 

The  final  configuration  of  the  Capture  Lab  central  table  required  multiple  iterations  of 
prototyping,  evaluation,  and  re-design.  The  result  was  a  unit  structure  coordinated  with  the 
room's  decor.  The  participants  sit  around  the  table  as  if  at  a  conventional  conference  table  —  facing 
inward  toward  the  center.  Their  Macintosh  displays  sit  squarely  before  them,  sunken  into  the  table 
surface  at  a  fixed  angle.  To  the  left  of  each  display,  a  panel  of  the  table  surface  may  be  lifted  to 
expose  a  recessed  cavity  in  which  each  participant's  mouse  and  a  floppy  disk  drive  are  stored. 
Individual  keyboards  are  stored  in  drawers  beneath  the  table.  This  "hiding"  of  the  storage  cavities, 
the  unit  table  surface,  and  even  the  use  of  compact  Apple  IIGS  keyboards  all  contribute  to  the 
intended  maximization  of  usable  table  space.  The  overall  effect  was  an  enhanced  conference  table, 
with  plenty  of  room  for  '  low-tech"  accessories  (documents,  files,  etc.).  Generally  speaking,  the 
Capture  Lab's  approach  emphasized  features  finely  tuned  to  the  characteristics  of  the  meeting  room 
(Docherty,  1992). 

In  contrast,  the  CSMIL  Collaboration  Technology  Suite's  table  is  actually  a  collection  of 
modular  computer  desks  (called  ELMERs)  developed  by  Steelcase  Corporation  (one  of  the 
facility's  corporate  sponsors).  These  units  were  designed  to  provide  maximum  flexibility  across 
many  conceivable  room  layouts  and  workstation  display  units.  To  that  end,  they  feature  (1) 
polygonal  desktop  surfaces,  so  that  they  can  be  arranged  in  groups  of  varying  si2^  and  angular 
orientation  and  (2)  motorized  supports  for  their  large  CRT  units.  These  supports,  controlled  by 
foot  switches,  can  position  the  CRT  horizontally  (flush  with  the  desktop  surface),  vertically,  or 
anywhere  in  between.  CSMIL  outlined  specifications  for  the  ELMERs  allowing  them  to  handle  a 
variety  of  computer  equipment  —  particularly  a  wide  range  of  monitors. 

Such  flexibility  has  been  obtained  at  a  cost.  The  ELMERs  are  heavy  (i.e.,  not  easily  moved 
about  to  take  advantage  of  the  promised  flexibility),  and  due  to  the  foot  switches  leg  room  is 
uncomfortably  constrained.  The  design  for  (and  use  of)  the  largest  available  monitors  results  in 
much  desktop  space  being  lost.  Finally,  the  polygonal  shape  of  the  desktop  surfaces  reduces  the 
area  available  for  use  when  the  desk  units  are  joined  in  a  "square"  arrangement.  At  the  point(s) 
where  the  units  meet  (i.e.,  the  "comers")  the  polygonal  shape  results  in  open  gaps  in  the  composite 
desk  surface.  Even  though  maximum  available  table  space  (e.g.,  for  papers)  was  cited  as  a 
desirable  feature,  the  Steelcase  design  was  deficient  in  this  regard.  Generally  speaking,  the 
CSMIL  approach  to  group  work  surfaces  has  emphasized  flexibility  and  functionality. 

A  second  example  (concerning  flexibility  of  information  display  rather  than  physical  layout)  can 
be  seen  in  each  facility's  attitudes  toward  whiteboards  as  group  displays.  At  the  EDS  Capture  Lab, 
a  pair  of  whiteboards  were  provided  on  the  wall  opposite  the  group  electronic  displays.  Mounted 
within  a  wall  niche,  they  were  normally  hidden  behind  wooden  panels.  As  reported  in  Docherty 
(1992)  the  use  of  whiteboards'  was  discouraged  in  relation  to  use  of  the  electronic  displays.  At  the 


CSMIL  Collaboration  Technology  Suite,  two  entire  walls  of  the  meeting  room  were  made  up  of 
whiteboard  panels,  and  participants  were  not  discouraged  from  using  them  as  they  saw  fit.  In  this 
respect,  the  CSMIL  facility  seemed  to  be  an  environment  more  suited  to  open-ended 
"brainstorming."  In  contrast,  the  EDS  Capture  Lab  imparts  a  degree  of  focus  with  respect  to  the 
electronic  support  tools. 

Physical  architecture  issues  don't  end  with  the  deployment  of  information  technology  and  other 
meeting  tools.  Both  facilities  reported  significant  problems  with  cables  and  connections  -  e.g., 
numbers  of  cables,  interference  among  lines,  inflexibility  with  regard  to  (re-)arrangement,  and  how 
to  hide  cables.  In  the  CSMIL  facility,  cable  layouts  restricted  (re-)configuration  options  for  the 
ELMER  units.  Because  multiple  units  could  not  be  easily  or  quickly  rearranged,  the  CSMIL 
researchers  were  not  able  to  take  full  advantage  of  their  intended  flexibility.  Another  problem 
concerned  the  length  of  cable  required  to  connect  the  workstations  with  the  file  server(s)  and  other 
attendant  units  in  an  adjoining  room.  The  CSMIL  facility  ended  up  paying  a  large  fee  for  dedicated 
cables  (especially  for  their  video  leads),  while  at  the  EDS  Capture  Lab  a  resident  "hacker"  spent 
much  time  coming  up  with  a  solution.  Our  experience  in  CDT  Lab  confirms  the  experience  at  these 
other  two  facilities.  We  had  to  regularly  rely  on  local  technical  staff  to  overcome  connectivity 
problems.  Our  laboratory'  space  had  suspended  floors  and  ceilings  which  facilitated  laying  and 
rearranging  cables  as  needed,  and  we  needed  to  do  this  on  a  regular  basis. 

As  reported  in  Docherty  (1992),  EDS  Capture  Lab  had  significantly  changed  in  its  short 
history,  but  staff  members  indicated  their  facility's  architecture,  though  sophisticated,  was  still  a 
prototype.  Their  conclusion  was  that  "when  you  build  one  [meeting]  room,  you'd  letter  be 
building  a  second  one"  -  both  to  allow  for  evolution  and  to  promote  research  opportunities.  We 
believe  that  (1)  integrated  group  interface  capacities  must  be  based  on  attention  to  such  physical  / 
architectural  factors  and  (2)  no  single  physical  group  workspace  architecture  is  likely  to  satisfy  ^ 
users  at  all  times.  Our  CDT  Lab  planning  was  therefore  directed  at  providing  for  flexibility  in 
terms  of  both  situational  usage  and  long-term  evolution. 

CDT  Lab  Physical  Architecture 

In  September  1993,  the  CDT  Lab  occupied  its  allotted  physical  facilities,  which  included 
researchers'  workspaces,  a  group  meeting/research  space,  and  auxiliary  space  for  technical 
support  /  office  expansion  /  observation  deck  development.  Some  furnishings  were  transferred 
from  the  previous  AKADAM  worksite,  and  the  remainder  (in  fact  the  majority)  were  assembled  by 
recycling  office  and  laboratory  furniture  from  our  own  and  adjacent  buildings  at  Wright-Patterson 
AFB.  Our  location  within  a  large  military  site  (with  a  well-developed  inventory  recycling  effort) 
enabled  us  to  furnish  our  new  work  and  research  space  at  what  was  effectively  no  net  cost.  Owing 
to  the  scale,  regular  turnover,  and  uniformity  of  the  site's  equipment  inventory,  we  found  that  with 
some  patience  it  was  possible  to  obtain  comparable  or  matched  items  such  as  chairs  and  tables. 

Approximately  one-third  of  the  CDT  Lab's  physical  space  was  dedicated  to  the  Group 
Workspace  -  a  meeting  and  collaboration  area  equipped  so  as  to  be  flexibly  reconfigurable  as 
needed.  The  Group  Workspace  served  as  a  meeting  room  for  CDTeam,  a  working  site  for 
facilitating  clients'  tasks,  a  demonstration  /  presentation  venue  for  CDT  work,  an  experimental  site 
for  CDT  studies,  a  prototyping  venue  for  group  workspace  configurations,  and  auxiliary  working 
space  for  guest  researchers  and  interns. 

The  Group  Workspace  was  approximately  15  feet  by  25  feet  in  size,  with  an  8-foot  ceiling. 
The  ceiling  and  floor  were  composed  of  suspended  tiles,  flowing  wires  and  other  equipment  to  be 
flexibly  mounted  and  reconfigured.  The  furnishings  consisted  of: 

•  (1)  Moveable  computer  work  desk  with  chair 

•  (1)  Fixed  table  supporting  the  LCD  projection  system 


•  (7)  Wall-mountable  whiteboards  ranging  in  size  from  30"  x  40"  to  4  feet  x  8  feet 

•  (1)  Free-standing  whiteboard  (pivotable  to  provide  double  its  4  feet  x  6  feet  area) 

•  (1)  60"  X  80"  projection  screen  mountable  on  the  wall 

•  (2)  Work  tables  which  could  be  arranged  as  needed  (e.g.,  one  square  conference  table;  two 

panels) 

•  A  variety  of  office  or  conference  room  chairs 

With  the  exception  of  the  projection  screen,  all  the  Group  Workspace  furnishings  were 
obtained  by  recycling  equipment  already  available  locally.  The  various  pieces  of  furniture  were  re¬ 
arranged  as  necessary  to  fit  the  number  of  participants,  the  type  of  meeting,  and  other  factors. 
Cameras,  microphones,  and  other  equipment  were  installed  or  deployed  on  an  "as-needed"  basis. 

IT  Infrastructure 

The  initial  CDT  Lab  configuration  included  7  Apple  Macintoshes,  divided  into  5  main 
workstations  and  2  smaller  Macs  for  auxiliary  use.  The  main  workstations  were  linked  into  a 
lOBase-T  Ethernet  LAN  and  (via  our  local  VAX  server)  to  the  Internet.  Two  additional  terminals 
provided  direct  access  to  the  VAX  server.  We  planned  for  a  variety  of  innovative  input  /  display  / 
control  devices  (e.g.,  the  Apple  Newton™  as  a  pen-based  personal  interface  unit).  Public  domain, 
shareware,  and  commercial  software  tools  (e.g.,  ShrEdit  and  Aspects™)  provided  collaborative 
writing,  brainstorming,  and  drawing  capabilities.  Other  groupware  products  were  occasionally 
acquired  by  or  referred  to  CDT  Lab  for  usability  assessment.  Additional  CDT  software  included 
concept  mapping  tools  deriving  from  the  earlier  AKADAM  work,  the  MacSHAPA  tool  for 
exploratory  sequential  data  analysis  (developed  in  part  with  Armstrong  Laboratory  support),  and  a 
wide  array  of  mainstream  software  applications  and  utilities.  Our  group  display  projection  system 
was  comprised  of  a  Sharp  QA-1650  color  LCD  projection  panel  and  a  high-intensity  DuKane  4003 
overhead  projector. 

During  the  winter  of  1993-1994,  the  Human  Engineering  Division  shifted  from  the  centralized 
VAX  server  to  a  microcomputer-based,  decentralized  LAN  architecture  (Novell  NetWare™  and 
WordPerfect  Office™  /  GroupWise™).  At  the  close  of  this  transition,  the  direct  terminals  to  the 
VAX  server  were  disconnected,  and  the  CDT  Lab  Macintosh  workstations  (now  equipped  with 
TCP/IP  software)  were  capable  of  direct  Internet  access.  At  that  point,  the  CDT  Lab  workstations 
were  configured  with  publicly  available  software  packages  such  as  TurboGopher  and  NCSA 
Mosaic.  This  set  the  stage  for  wider  monitoring  of  relevant  developments  in  CSCW,  networking, 
and  the  like,  thus  augmenting  CDTeam's  abilities  to  remain  abreast  of  the  state  of  the  art.  It  also 
set  the  stage  for  evaluation  of  tools  for  creating  and  maintaining  World  Wide  Web  materials  using 
the  HyperText  Markup  Language  (HTML). 


Video  Link 

The  CDT  Lab  was  the  Human  Engineering  Division’s  second  collaboration  facility  and  CSCW 
research  testbed.  The  Division's  Multimedia  Room  is  an  operational  meeting  /  presentation  site 
with:  an  electronic  whiteboard,  large  group  video  display,  video  equipment,  Macintosh  Quadra, 
multimedia  support  software,  35mm  slide  and  overhead  projectors,  teleconferencing  equipment, 
and  visualizer.  CDT  Lab  installed  video  cameras  in  the  Multimedia  Room  to  allow  monitoring  and 
recording  of  meetings.  Attached  to  the  Multimedia  Room  was  a  control  room  with  extensive  audio 
and  video  recording  and  editing  equipment.  This  facility  was  employed  in  and  of  itself  for 
empirical  studies  of  collaborating  real-world  design  professionals  (cf  the  later  section  on  the 
TRACE  experiments  and  Brown,  Selvaraj,  Whitaker,  and  McNeese  (in  press)). 

The  plan  for  CDT  Lab's  research  infrastructure  extended  the  Multimedia  Room's  rote  beyond 
simply  being  a  second  experimental  site.  Video  teleconferencing  (both  desktop-to-desktop  and 
room-to-room)  was  a  recurring  item  of  interest  for  CDT  Lab's  sponsors  and  clients.  We  realized 
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that  the  typically  high  ramp-up  costs  for  video  teleconferencing  could  be  reduced  to  a  tractable  level 
by  exploiting  access  to  the  Multimedia  Room's  existing  video  support  equipment  (e.g.,  editing 
tools;  mixers;  switching  panels).  Furthermore,  we  recognized  that  linking  the  Multimedia  Room 
with  the  CDT  Lab  would  provide  us  a  testbed  within  which  to  test  video  conferencing 
technologies.  Precedents  for  this  sort  of  "in-house"  video  testbed  include  the  Decision  Room  at 
Southern  Methodist  University  and  the  Decision  Laboratory  at  the  Claremont  Graduate  School 
(Gray,  1992).  The  former  was  not  finally  implemented,  and  the  latter  was  only  recently  reaching 
operational  status. 

During  the  spring  and  summer  of  1994,  we  constructed  a  multichannel  data  /  audio  /  video  link 
between  the  Multimedia  Room's  control  room  and  the  CDT  Lab.  On  the  Multimedia  Room  end, 
this  link  was  tied  into  the  existing  control  room  equipment.  On  the  CDT  Lab  end,  the  link 
terminated  in  an  array  of  control  /  recording  equipment  assembled  from  available  local  inventory 
(e.g.,  used  or  remainder  items  in  storage).  By  making  the  CDT  end  of  the  video  link  100% 
"recycled"  equipment,  we  further  minimized  the  start-up  costs  for  establishing  a  video 
conferencing  testbed.  These  two  linked  rooms  provided  us  a  single-site  capacity  to  simulate  all 
dyadic  permutations  of  in:'ividual  /  group  interactivity  in  remotely  distributed  scenarios. 
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CDT  RESEARCH  APPROACHES 


Overview  /  Summary  of  Issues 

As  described  earlier,  tbe  CDT  Lab's  research  foci  entailed  covering  a  wide  range  of  topical  and 
analytical  territory.  Not  surprisingly,  this  diversity  of  issues  and  interests  was  determined  to 
exceed  the  ability  of  any  one  research  approach  to  manage.  Based  on  theoretical  concerns,  prior 
experiences,  and  circumstances,  we  subdivided  our  research  activities  into  four  basic  categories: 

•  Experimental  studies  in  which  we  monitored  and  analyzed  subjects'  actions  within 

settings  configured  in  conformance  to  research  paradigms  carefully  contrived  to  test 
issues  of  collaboration  in  design  (particularly  the  identification  and  negotiation  of 
design  tradeoffs).  This  approach  can  be  characterized  as  conducting  artificial  tasks  in 
artificial  settings  toward  the  goal  of  testing  known  or  suspected  parameters  of 
collaborative  behavior.  The  experimental  studies  addressed  the  "CD"  (collaborative 
design)  leg  of  the  CDT  Triangle. 

•  Observational  studies  in  which  we  either  (1)  observed  design  professionals  in  the 

course  of  their  actual  design  collaborations  or  (2)  elicited,  compiled,  and  analyzed 
design  professionals'  self-reports  of  critical  success  factors  for  collaboration  in 
multidisciplinary  design  teams.  This  approach  can  be  characterized  as  carefully 
monitoring  actual  tasks  in  their  natural  settings  toward  the  goal  of  identifying  new  or 
affirming  prior  parameters  of  collaborative  behavior.  The  observational  studies 
addressed  the  "CD"  (collaborative  design)  leg  of  the  CDT  Triangle. 

•  Simulation  studies  in  which  we  monitored  and  analyzed  the  behavior  of  design  (or 

other)  professionals  within  settings  wherein  one  or  more  human  factors  aspects  of 
information  support  or  technology  were  themselves  the  subject  of  interest.  This 
approach  can  be  characterized  as  conducting  natural  tasks  within  settings  where  the 
functional  relatioi.  ships  between  the  participants  and  their  support  tools  were 
manipulated  toward  the  goal  of  identifying  new  or  affirming  prior  parameters  of 
collaborative  behavior.  The  simulation  studies  addressed  interactions  between  the 
"CD"  (collaborative  design)  and  "CT"  (collaborative  technology)  legs  of  the  CDT 
Triangle. 

•  Technology  studies  in  which  we  either  (1)  evaluated  groupware  and  other  relevant 

technical  artifacts  or  (2)  developed  novel  technology  addressing  issues  related  to  the 
other  three  categories  of  research.  The  technology  studies  addressed  the  "CT" 
(collaborative  technology)  leg  of  the  CDT  Triangle. 

Figure  3  illustrates  how  these  four  types  of  research  activity  relate  to  the  issues  delineated  in  the 
"CDT  Triangle"  (cf.  Figure  2).  Ours  was  a  two-stage  program.  First,  we  sought  to  better 
understand  collaborative  design  activities  (Track  lA)  and  technological  support  for  them  (Track 
IB).  The  second  stage  was  to  carry  forward  research  results  to  constructively  improve  design 
technologies  (processes  and  tools). 

In  the  following  sections,  we  shall  describe  each  of  the  four  research  categories  in  more  detail, 
illustrating  our  progress  along  Tracks  lA  and  IB  of  the  research  plan  with  specific  examples. 
Some  of  the  examples  represent  work  which  has  been  already  reported  in  the  literature  or  will  soon 
appear  in  the  literature.  In  those  cases,  we  shall  provide  references  for  the  reader  interested  in 
following  up  on  the  information  in  this  report.  Some  of  the  examples  represent  studies  involving 
teams  of  CDT  Lab  clients  within  the  USAF.  Owing  to  confidentiality  constraints  deriving  from 
USAF  practices  generally  or  particular  agreements  between  CDTeam  and  clients,  we  shall  not 
identify  clients  more  specifically  than  by  programmatic  affiliation  nor  necessarily  detail  the  subject 
matter  of  their  collaboration. 
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COLLABORATION 


lA 

study  Collaborative  Design  via: 

•  Experimental  Studies  of  design 
collaboration  parameters  using 
structured  paradigms 


Observational  Studies  of 
actual  design  teams  in 
their  activities 


Study  Collaborative  Technology  via: 

•  Simulation  Studies  of  actual  design 
teams'  usage  of  and  interaction 
with  information  technology 

•  Evaluations  of  available 
groupware  and  other 
relevant  tools 


DESIGN 


Technology 

2 

Leverage  Design  Technology 
by  innovatively  applying 
the  results 


TECHNOLOGY 


Figure  3:  The  CDT  Lab's  issue-oriented  research  strategy 


CDT  Lab  Experimental  Studies 

The  experimental  studies  comjwnent  of  the  CDT  Lab  agenda  was  intended  to  produce  empirical 
investigations  of  collaborative  design  activities.  This  mode  of  enquiry  focused  on  the  conduct  of 
test  subjects  within  a  contrived  task  setting.  The  construction  of  such  tasks  (or  paradigms)  was 
therefore  of  great  importance.  At  the  time  of  the  CDT  Lab's  inauguration,  Dr.  Clifford  Brown 
(visiting  researcher  from  Wittenberg  University)  had  finalized  the  TRACE  paradigm  addressing 
design  tradeoff  negotiations  (cf.  Brown,  Selvaraj,  Whitaker,  and  McNeese  (in  press)),  and  Dr. 
Maryalire  Citera  (AFOSR-sponsored  visiting  researcher  from  Wright  State  University)  was 
developing  the  AutoMate  paradigm  addressing  design  information  sharing.  Additional  frameworks 
previously  employed  by  the  authors  and  readily  available  for  CDT  usage  included  the  TRAP 
experimental  paradigm  previously  used  in  studying  multi-operator  collaborative  activities  (Brown 
and  Leupp,  1985)  and  the  Jasper  paradigm  developed  to  address  situated  problem  solving 
(McNeese,  1993). 

TRACE 

Effective  design  requires  design  decisions  integrating  relevant  information  from  a  variety  of 
sources  (Boff,  1987).  Complex  system  design  entails  negotiation  among  parties  pursuing 
common  goals  with  potentially  divergent  interests  and  objectives  (Bucciarelli,  1988).  In 
multidisciplinary  design  teams,  these  parties  negotiate  from  perspectives  biased  by  their  respective 
backgrounds,  expertise,  and  roles.  By  impeding,  effective  communication  and  preventing  open, 
fully-informed  consideration  of  design  issues,  these  disjunct  outlooks  effect  “cross-disciplinary 
chokepoints”  (e.g.,  unawareness  of  or  blindness  to  relevant  technical  data  —  Boff,  1987).  These 
chokepoints  are  problematic  even  for  the  relatively  straightforward  case  of  assessing  tradeoffs 
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among  a  product's  functions  and  attributes,  which  can  be  reduced  to  a  dichotomous  decision  to 
include  or  exclude  a  sp^ific  feature  (Cody,  Rouse,  and  Boff,  1993).  In  a  multidisciplinary 
setting,  negotiating  such  inclusion  becomes  contingent  upon  effectively  communicating  one's  own 
criteria,  effectively  assimilating  others'  criteria,  and  effectively  cooperating  to  arrive  at  a  mutually 
advantageous  solution. 

As  part  of  our  larger  research  effort  in  Collaborative  Design  Technology,  we  have  examined  the 
processes  by  which  integrative  design  tradeoffs  are  realized,  in  preparation  for  enhancing  these 
processes  through  data  visualization  and  communication  tools  facilitating  mutual  understanding  and 
consensual  decision  making.  Integrative  bargaining  research  in  social  psychology  has 
systematically  explored  development  of  mutually  advantageous  outcomes  in  bilateral  negotiation, 
t^ically  using  students  role-playing  buyers  and  sellers  (Pruitt  and  Lewis,  1975).  In  order  to 
explore  the  applicability  of  this  research  to  multidisciplinary  design,  we  constructed  an  integrative 
bargaining  paradigm  (TRACE  —  Tradeoffs,  Research,  and  Analysis  in  Collaborative  Ergonomics) 
framed  with  respect  to  design  tradeoffs  in  the  development  of  an  automobile  navigation  system 
(Citera  and  Selvaraj,  1992;  McNeese,  et  al.,  1993). 

Because  of  our  role  as  a  human  factors  laboratory  supporting  U.S.  Air  Force  operations,  we 
emphasize  ecological  validity  (Brunswik,  1956)  in  our  experimentation.  This  raises  the  question 
of  how  to  assess  our  contrived  research  paradigm's  validity  with  respect  to  real-world  design  —  a 
research  issue  in  itself.  We  hypothesized  that  if  the  paradigm  reasonably  captures  the  context  of 
real-world  multidisciplin^  design  practice,  good  task  performance  should  be  demonstrably 
projwrtional  to  actual  design  experience.  Our  experimental  results  support  this  hypothesis.  More 
detailed  descriptions  of  both  the  TRACE  paradigm  and  results  of  its  employment  to  date  can  be 
found  in  Brown,  Selvaraj,  Zaff,  McNeese,  and  Whitaker  (1994)  and  Brown,  Selvaraj,  Whitaker, 
and  McNeese  (in  press). 

AutoMate 


AutoMate  is  a  research  paradigm  devised  to  provide  a  framework  for  experimentally  studying 
collaboration  in  multidisciplinary  design.  The  paradigm  was  specifically  crafted  to  address 
concurrent  engineering's  problems  with  miscommunication,  miscoordination,  and  misanalogies,  to 
the  extent  they  derive  from  suboptimal  information  sharing.  When  design  is  conducted  in  a  team 
setting,  the  mass  of  available  relevant  knowledge  is  distributed  among  a  potentially  wide 
population.  When  this  population  spans  a  variety  of  disciplines,  the  form  of  this  knowledge  may 
be  "distributed"  among  a  similar  variety  of  jargons,  models,  background  assumptions,  etc.  Owing 
to  their  diverse  backgrounds,  training,  and  professional  practices,  team  members  may  effectively 
appear  to  speak  different  languages  (Boff,  1987).  The  resultant  misunderstandings  and 
misinterpretations  constitute  miscommunications. 

In  addition,  multidisciplinary  team  members  may  work  on  different  schedules  and  be  separated 
by  physical  distances.  These  factors  may  impede  not  only  the  actual  design  workflow  (e.g., 
discussions,  decisionmaking,  and  actions),  but  also  the  coordination  of  the  workflow.  Anyone 
who  has  played  telephone  tag  understands  the  frustration  of  being  unable  to  locate  another  team 
member.  Such  miscoordination  may  result  in  team  members  (e.g.)  failing  to  fully  inform  each 
other,  making  decisions  based  on  incomplete  information,  and  generally  wasting  their  time  and 
effort. 

Furthermore,  designers  often  rely  on  a  case-based  strategy  (Gero,  1990;  Klein,  1987)  ~ 
approaching  the  current  design  problem  by  drawing  analogies  from  past  designs  instead  of 
generating  and  carefully  evaluating  all  possible  current  alternatives.  The  advantage  of  a  case-based 
approach  is  that  good  fe<'.tures  from  previous  designs  can  be  readily  incorporated  into  the  new 
project.  On  the  other  hand,  if  components  of  an  earlier  design  are  spuriously  considered  essential 
for  —  and  included  in  —  the  current  case,  their  unwarranted  inclusion  may  restrict  the  design 
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options  considered.  When  previous  learning  is  applied  to  a  situation  where  it  may  be  inappropriate 
or  conflict  with  other  aspects  of  the  design,  misanalogies  occur.  Because  misanalogies  are  difficult 
to  articulate,  they  add  to  the  communication  problems  multidisciplinary  design  teams  face. 

Team  members  must  rely  on  both  their  own  unique  information  and  that  of  other  team  members 
to  inform  and  guide  themselves  throughout  the  design  process.  Unfortunately,  multidisciplinary 
design  teams  do  not  effectively  share  distributed  knowledge.  According  to  Stasser  (1992);  Stasser 
and  Titus,  1985;  1987),  groups  often  focus  on  shared  information  and  neglect  to  discuss  unique  or 
unshared  information.  This  may  be  because  (e.g.)  members  of  the  team  do  not  share  an  effective 
common  framework  within  which  design  knowledge  can  be  conveyed  (miscommunication), 
cannot  effectively  transfer  their  data  or  otherwise  pursue  the  design  activity  (miscoordination),  or 
cannot  relate  the  data  to  the  problem  at  hand  (misanalogy).  In  order  to  study  these  aspects  of 
information  sharing,  the  AutoMate  task  was  created. 

Based  on  Stasser's  (1992)  hidden  profile  paradigm,  AutoMate  requires  subjects  to  consider  the 
design  of  an  automobile  navigation  display.  Each  subject  receives  a  body  of  relevant  data  (the 
shared  information)  as  well  as  a  guidebook  and  design  rationale  which  are  specific  to  his/her  own 
disciplinary  role  (the  unique  information).  The  construction  of  the  specific  design  scenarios  and 
the  information  packages  relevant  to  them  have  represented  the  primary  challenges  in  developing 
AutoMate.  By  manipulating  the  specific  conditions  for  an  automobile  navigation  design  exercise  as 
well  as  the  distribution  of  information  necessary  to  its  accomplishment,  AutoMate  provides  a 
realistic  task  framework  within  which  patterns  and  degrees  of  information  sharing  can  be  tracked. 
Information  sharing  can  be  assessed  in  terms  of  (1)  how  much  of  the  shared  information  is  applied 
and  (2)  how  much  of  the  unique  information  is  presented,  shared,  and  incorporated  in  the  final 
outcome. 

The  design  domain  for  the  AutoMate  task  -  automobile  navigation  displays  -  is  based  on 
actual  technical  data  and  experiences  obtained  in  part  from  design  professionals  (cf.  the  ASC 
Collaboration  Study  described  elsewhere  in  this  report).  This  affords  the  task  a  measure  of 
ecological  validity  (Brunswik,  1956).  Because  the  automobile  navigation  design  domain  is  the 
same  one  used  in  the  TRACE  experimental  paradigm,  we  have  ensured  some  degree  of  uniformity 
across  the  tasks.  The  CDT-sponsored  work  toward  developing  the  AutoMate  paradigm  is 
discussed  in  more  detail  in  Citera  and  Selvaraj  (1992),  Citera  et  al.  (1993),  and  the  companion 
technical  report  (Citera,  Selvaraj,  McNeese,  Brown,  &  Zaff,  in  press). 

CDT  Lab  Observational  Studies 

Both  applied  and  basic  research  efforts  in  the  CSCW  field  have  increasingly  turned  toward 
ethnography,  ethnomethodology,  qualitative  research,  and  other  methods  emphasizing  researchers' 
observations  of  collaboration  in  situ  and  a  suspension  of  theoretical  framing  pending  the  collection 
of  extensive  empirical  data.  Related  terminology  includes:  grounded  theory,  case  studies, 
conversational  analysis,  discourse  analysis,  and  action  research.  Weick  (1984)  concisely 
summarized  such  approaches  under  the  label  intensive  research,  based  on  the  intensive  study  of 
relatively  few  subjects  (as  opposed  to  a  narrowly-focused  study  of  many  subjects).  Such  studies 
typically  provide  valuable  data  for  framing  research  issues  and  specifications  for  development 
(e.g.,  Reder  &  Schwab,  1988;  1990).  Ethnography  derives  from  anthropological  field  techniques 
which  avoid  reliance  on  quantitative  data  collection  in  favor  of  "...a  commitment  to  a  period  and 
degree  of  immersion  in  the  social  setting  being  studied  that  is  sufficient  to  reach  a  qualitative 
understanding  of  what  happens  there."  (Shapiro,  1994,  p.  418).  The  subset  of  ethnography  most 
prevalent  in  CSCW  research  is  ethnomethodology,  initially  formulated  by  Garfinkel  (e.g.,  1967) 
and  popularized  in  IT  circles  by  the  work  of  Lucy  Suchman  (e.g,  1987).  Ethnometihodology  is 
rigorously  empirical  in  the  sense  that  it  prioritizes  comprehensive  data  collection  as  the  prologue  to 
analysis  and  theory  building,  in  contrast  to  the  theory-driven  nature  of  other  qualitative  approaches. 
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The  most  tangible  motivation  for  pursuing  ethnography  in  groupware  design  is  the  explicit 
intent  and  widespread  acknowledgement  that  distributed,  communicative  technologies  intervene  in 
workplace  collaboration  (as  opposed  to  simply  intervening  in  individual  functions  or  operations). 
As  such,  these  technologies'  deployment  affects  the  interpersonal  or  social  aspects  as  well  as  the 
inter-role  or  functional  aspects  of  work.  These  social  aspects  of  work  can  only  be  assessed  on 
their  own  terms,  because  they  lie  outside  the  scope  of  conventional  (functionalist)  requirements 
capture  and  analysis.  Overlooking  such  social  aspects  of  collaboration  has  been  cited  as  the 
recurrent  critical  factor  in  groupware  implementation  failures  to  date  (Grudin,  1988;  1990a; 
1990b). 

This  does  not  mean  that  ethnography  is  easily  applied  to  systems  design.  The  leading 
practitioners  of  ethnography  in  design,  Hughes  et  al.  (1994),  cite  three  major  problems  in 
transitioning  ethnographic  methods  from  their  academic  research  origins  into  workaday 
applications.  First,  ethnographic  methods  work  best  for  relatively  small  and  distinct  groups.  In 
moving  to  larger  and  /  or  less  well-delineated  groups  (e.g.,  hundreds  of  workers  in  a  project 
organization),  one  runs  into  problems  of  scale.  Second,  ethnographic  studies  in  the  social  sciences 
are  typically  of  long  duration  —  sometimes  lasting  years.  This  time-intensive  nature  may  apply  to 
both  the  observational  and  the  analytical  phases  of  ethnographic  research.  In  moving  to  more  time- 
constrained  or  deadline-driven  settings  (e.g.,  systems  design  projects),  one  runs  into  problems  of 
time  pressure.  Finally,  ethnographic  studies  in  the  social  sciences  are  typically  descriptive 
academic  exercises  whose  conduct  and  analyses  are  relatively  non-threatening  to  the  observed 
subjects.  In  moving  to  commercial  and  other  development  settings,  access  to  and  description  of 
the  subject  activities  become  more  problematic  due  to  (e.g.)  proprietary  constraints,  ethical 
considerations,  and  fear  of  negative  feedback.  In  other  words,  there  are  problems  of  the 
ethnographer's  role. 

Ethnography's  emphasis  on  studying  work  within  its  workplace  context  related  to  the  concern 
for  ecological  validity  CDT  Lab  inherited  from  the  earlier  AKADAM  efforts  (cf.  McNeese  et  al. , 
1993).  The  term  "ecological  validity"  was  coined  by  the  ecological  psychologist  Egon  Brunswik 
(1956)  to  denote  the  degree  to  which  an  experimental  scenario  corresponded  to  the  actual  or 
"natural"  setting  for  the  phenomenon  or  behavior  being  tested.  In  contrast  to  experimental 
approaches  to  studying  group  processes  in  collaborative  design,  CDT  Lab  observational  studies 
emphasized  direct  field  observations  of  design  team  members  engaged  in  design  meetings. 

The  near-term  goal  of  the  CDT  observational  studies  was  to  explore  and  assess  ethnographic 
practices  for  their  appUcabihty  in  systems  design.  The  long-term  goal  of  this  approach  was  to 
examine  the  conduct  of  real-world  design  collaboration  to  identify  factors  relevant  to  effective 
configuration  of  support  tools  (e.g.,  information  technology).  The  method  was  to  collect 
background  (e.g.,  documentation)  and  situational  data  (e.g.,  minutes  of  meetings)  in  preparation 
for  analysis.  In  our  efforts  directed  at  the  long-term  goal,  these  observations  were  augmented  by 
elicitation  of  design  team  members’  experiences  in  design  collaboration  (both  the  observed 
meetings  and  other  collaborations).  Our  observational  studies  could  be  characterized  as 
concentrating  on  the  Collaborative  Design  (CD)  leg  of  the  "CDT  Triangle." 

In  the  following  sections,  we  will  present  and  discuss  two  CDT  Lab  observational  studies. 
The  first  —  involving  the  CCCD  Field  Demo  2  --  was  aimed  at  our  near-term  goal  of  evaluating 
ethnographic  methods.  It  emphasized  CDTeam  observation  of  a  design  team  within  its  working 
context.  The  second  —  involving  the  AutoMate  studies  —  was  more  directed  at  the  long-term  god 
of  identifying  factors  relevant  to  IT  support  for  systems  design.  It  emphasized  structured 
interviewing  surveying  the  experiences  of  designers  with  actual  experience  on  a  specific  design 
task. 
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CCCD  Field  Demo  2 


Introduction  and  background.  In  May  1994,  the  CDT  Laboratory  proposed  an  observational 
study  involving  the  Crew-Centered  Cockpit  Design  (CCCD)  Laboratory  in  Armstrong 
Laboratory’s  Human  Engineering  Division.  The  goals  of  this  study  were: 

(1)  To  familiarize  CDT  Lab  personnel  with  ethnographic  methods. 

(2)  To  assess  the  application  of  ethnographic  methods  in  a  systems  design  setting. 

(3)  To  assess  the  practicality  of  a  large-scale  ethnographic  study  of  either  the  CCCD  or  a 
similar  design  team. 

The  CCCD  Laboratory  was  in  the  process  of  conducting  validation  studies  of  its  structured 
methodology  and  tools  for  re-designing  aircraft  crewstations.  The  CCCD  methods  emphasized 
rapid  prototyping  and  hands-on  simulation  of  the  crewstation(s)  of  interest.  As  of  June  1994, 
CCCD  project  personnel  were  beginning  the  second  of  their  Field  Demo's  (validation  studies),  the 
subject  of  which  was  the  re-design  of  the  A/C  130  gunship  navigation  and  fire  control 
workstations.  A  multidisciplinary  design  team  consisting  of  software  and  hardware  engineers, 
human  factors  specialists,  subject  matter  experts,  and  program  managers  was  involved  in  the 
summer  1994  validation  effort  (hereafter  termed  Field  Demo  2)  -  making  it  ecologically  valid  with 
respect  to  USAF  systems  (re-)design  generally.  The  involvement  of  actual  flight  crew  members  as 
subject  matter  experts  (SME's)  and  participants  in  the  simulation  runs  afforded  reasonable 
ecological  validity  with  respect  to  the  particular  system  of  concern. 

We  had  determined  that  any  design  project  targeted  for  our  initial  ethnographic  data  collection 
should  be  short-termed  and  highly  focused,  so  that  immersion  into  the  design  ^tting  was  both 
operationally  feasible  and  conducive  to  substantive  data  collection.  The  CCCD  Field  I^mo  2  had 
an  expected  duration  of  approximately  3  months  and  was  concentrated  on  the  evaluatiori  and  re¬ 
design  of  two  aircraft  crewstations.  These  factors,  in  our  opinion,  satisfied  our  general  criteria. 

In  a  full-scale  ethnographic  study,  it  would  have  been  the  CDTeam's  goal  to  imnierse  itself  in 
the  daily  activities  of  a  working  design  team  and  coUect  data  via  (e.g.)  direct  observations,  activity 
logs,  video  records,  and  audio  recordings.  This  was  the  general  plan  offered  by  CDTeam  in  its 
initid  meetings  with  CCCD  staff  and  government  monitors.  However,  not  all  die  facets  of  this 
suggested  plan  were  implemented,  due  to  concerns  voiced  by  both  CCCD  staff  and  CDTeam.  The 
prospect  of  being  the  subjects  of  extensive  field  observations  immediately  raised  a  number  of 
questions  on  the  part  of  the  CCCD  personnel.  Some  of  the  issues  raised  included:  confidentiality 
of  observational  data;  degree  of  CDTeam  access  to  potentially  proprietary  information;  logistics  of 
CDTeam  access  to  the  CCCD  project  site;  logistics  of  CDTeam  access  to  Field  Demo  2  activities; 
and  logistics  of  CDTeam  access  to  participating  personnel.  In  parallel,  CDTeam  began  to  question 
the  feasibility  of  full-blown  ethnographic  data  collection  in  the  course  of  this  initial  exercise.  The 
Field  Demo  2  work  would  proceed  daily  and  involve  approximately  a  dozen  on-site  personnel. 
Comprehensive  video  and/or  audio  records  of  the  design  work  would  require  equipment  and  media 
stock  representing  a  potentially  excessive  investment. 

In  the  end,  CDTeam's  negotiations  with  the  CCCD  project  staff  resulted  in  a  data  collection 
plan  which  generally  limited  CDTeam  access  to  the  project  status  meetings  held  (on  a  weekly  basis) 
during  Field  Demo  2.  Video  recording  was  ruled  out,  and  audio  recordings  of  even  these  status 
meetings  were  not  permitted  at  first.  After  further  negotiation,  the  CDTeam  was  permitted  to  make 
audio  recordings  beginning  with  the  meeting  of  22  June  1994  and  continuing  throughout  the 
remainder  of  our  Field  Demo  2  observations.  In  addition  to  the  status  meetings,  CDTepi 
monitored:  the  design  of  data  collection  methods  for  evaluating  the  workstation  redesign 
(15JUL94),  a  two-day  Cockpit  Working  Group  (CWG)  meeting  in  which  subject  matter  experts 
(SME’s)  evaluated  prospective  design  changes  (26JUL94  /  27JUL94),  the  CCCD  outbriefing  for 
government  monitors  and  SME’s  (01SEP94),  and  the  CCCD  project  staffs  review  of  change 
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proposals  generated  during  the  Field  Demo  (07SEP94).  In  summary,  the  Field  Demo  2  meetings 
we  monitored  were  as  follows; 

17MAY94 
01JUN94 
07JUN94 
09JUN94 
16JUN94 
22JUN94 
29JUN94 
07JUL94 
15JUL94 
26JUL94 
27JUL94 
03AUG94 
16AUG94 
24AUG94 
31AUG94 
01SEP94 
07SEP94 

These  meetings  covered  the  progress  of  Field  Demo  2  from  its  inception  through  to  its 
conclusion  with  an  outbriefmg  with  the  SME's  and  a  review  of  proposed  changes  to  the  CCCD 
methodology  and  tools.  In  terms  of  the  general  temporal  extent  of  observational  coverage, 
CDTeam  obtained  adequate  access  to  the  CCCD  Field  Demo  2.  In  terms  of  the  "depth"  of 
observational  coverage,  CDTeam's  monitoring  of  Field  Demo  2  would  not  have  been  sufficient  for 
a  full-blown  ethnographic  study.  However,  for  the  stated  purposes  of  this  exercise  (familiarization 
with  and  assessment  of  ethnographic  practices),  the  relatively  "shallow"  data  collection  program 
negotiated  with  CCCD  staff  was  sufficient. 

For  each  of  the  listed  meetings,  one  or  more  CDTeam  observers  were  present.  The  maximum 
number  of  CDTeam  observers  in  attendance  was  four,  and  two  was  typical  for  most  of  the 
meetings.  The  main  method  of  data  collection  was  writing  notes,  augmented  starting  22  June  1994 
with  audio  taping  using  a  portable  cassette  recorder.  The  weekly  status  meetings  varied  from  30  to 
90  minutes  in  length,  the  outbriefmg  lasted  2  hours,  and  the  CWG  meetings  spanned  2  days.  After 
each  meeting,  the  observers  would  return  to  CDT  Lab  to  collate  their  written  notes  and  transcribe 
them  via  word  processor  into  computer  files,  which  were  then  collected  and  collated  by  a 
designated  lead  observer.  CDTeam  observers  would  often  compare  notes  for  clarification  during 
this  transcription  period. 

The  following  sections  summarize  the  results  and  lessons  learned  assoociated  with  our 
evaluation  of  the  CCCD  field  demo  2. 

Ethnographic  studies  require  a  great  deal  of  background  preparation.  A  real-world  design  team 
works  toward  its  own  goals  on  its  own  terms.  In  contrast  with  experimental  settings  wherein  all 
but  the  issues  and  factors  of  research  interest  have  been  referentially  neutralized,  observers  of 
actual  workplace  practice  must  be  prepared  to  interpret  their  subjects  in  a  more  complex  and  less 
regularized  setting.  Much  of  this  complexity  can  be  attributed  to  situational  factors  -  i.e., 
characteristics  of  the  work  which  are  s^cific  to  the  particular  setting.  In  highly  technical  settings 
such  as  systems  design,  situation-specific  factors  such  as  terminology,  documentation,  history  of 
prior  decisions,  local  codes  of  practice,  and  the  results  of  circumstantial  responses  contribute  to 
this  complexity.  We  found  this  to  be  the  case  in  Field  Demo  2. 

The  CCCD  design  team  provided  the  CDTeam  with  a  considerable  amount  of  written  material 


Introductory  Meeting  with  CCCD  Contractor  and  Government  Personnel 

Project  Status  Meeting 

Project  Status  Meeting 

Project  Status  Meeting 

Project  Status  Meeting 

Project  Status  Meeting 

CDTeam  Interim  Meeting  with  CCCD  Management  (background  data) 

Project  Status  Meeting 

Meeting  to  Design  Data  Collection 

Cockpit  Working  Group  Meeting,  Part  I 

Cockpit  Working  Group  Meeting,  Part  II 

Briefing  to  CCCD  Design  Team  on  Observations 

Project  Status  Meeting 

Project  Status  Meeting 

Observations  of  Field  Demo  2  Simulation  Runs  /  Data  Collection 
Outbriefing 

Change  Proposal  Review  Meeting 
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(conference  papers,  weekly  activity  reports,  and  technical  reports),  which  were  collated  and 
reviewed  as  background  material  in  support  of  the  observational  study.  This  material  proved 
critical  in  facilitating  our  ability  to  track  and  interpret  the  observed  meetings,  owing  to  the  design 
team's  reliance  on  the  CCCD  structured  "map"  for  the  Field  Demo  process  and  the  relatively  large 
amount  of  CCCD-specific  terminology  employed.  One  illustration  of  this  issue's  importance  was 
that  much  of  the  interim  meeting  with  CCCD  staff  management  (29JUN94)  concentrated  on 
CDTeam's  confirming  interpretations  of  CCCD-specific  items. 

Ethnographic  studies  entail  considerable  expense  for  data  collection.  As  noted  earlier,  one  of 
the  reasons  for  scaling  down  the  Field  Demo  2  exercise  from  the  initial  suggestion  of  a  full-blown 
ethnographic  effort  was  the  cost  for  equipment  and  media  stock  requisite  to  simply  recording  the 
design  activities.  The  fact  that  we  determined  the  projected  costs  were  exorbitant  for  this 
familiarization  exercise  does  not  imply  that  they  are  automatically  manageable  for  full-scale  studies. 
Labor  costs  for  observation  are  directly  proportional  to  the  number  of  observers  per  session  and 
the  duration  of  the  sessions  observed.  Based  on  our  typical  practice,  total  coverage  of  Field  Demo 
2  would  have  entailed  a  minimum  of  2  CDTeam  observers  over  the  entire  3-month  period  -  a  total 
of  0.5  person-years  and  approximately  15-20%  of  the  person-years  invested  by  the  Field  Demo  2 
team  itself  in  conducting  their  work. 

Ethnographic  studies  entail  considerable  expense  for  data  collation.  The  simple  transcription  of 
observer  notes  into  electronic  files  with  a  word  processor  consistently  took  at  least  twice  the 
amount  of  time  taken  up  by  the  meeting  in  which  Ae  notes  were  made.  For  individual  observers 
transcribing  their  own  notes,  the  minimum  transcription  ratio  (ratio  of  transcription  time-to- 
observation  time)  we  noted  was  approximately  1.5:1,  and  the  maximum  noted  was  approximately 
5:1.  We  attempted  to  streamline  this  process  by  having  multiple  observers  cooperatively  compile 
and  transcribe  all  the  notes  from  a  meeting.  The  result  was  a  marked  degradation  in  the 
transcription  ratio.  In  one  instance  the  collaborating  transcribers  were  the  CDTeam's  fastest  on  an 
individual  basis  (consistently  2:1  or  better),  but  managed  only  about  a  4:1  transcription  ratio 
working  jointly.  For  both  individual  and  joint  transcriptions,  we  found  that  much  time  ended  up 
being  invested  in  interpreting  raw  meeting  notes  (one's  own  or  someone  else's),  clarifying  points 
by  consultation  with  others,  and  "fleshing  out"  the  resultant  text.  Trial  attempts  to  augment 
transcription  with  the  audio  tapes  did  not  result  in  a  noticeable  improvement  in  the  transcription 
ratio.  As  a  result,  the  audio  tapes  remained  relegated  to  a  backup  role,  providing  a  comprehensive 
reference  of  last  resort  on  those  occasions  where  individuals'  notes  were  contradictory  or 
incomplete. 

Ethnographic  studies  entail  a  reversal  of  relative  procedural  control  between  researcher  and 
subject(s).  In  an  experimental  setting,  researchers  control  the  scheduling  and  duration  of  sessions, 
ma^ng  their  time  and  resource  allocation  relatively  easy.  In  ethnographic  studies  with  total 
observational  coverage,  researchers  are  obligated  to  be  on  hand  wherever  and  whenever  the 
subjects  are  active.  This  means  that  ethnographic  researchers  are  at  the  mercy  of  the  subjects’ 
scheduling  constraints,  breakdowns,  and  other  coordination  problems.  In  addition  to  the  obvious 
potential  for  inconvenience,  this  can  affect  researchers'  ability  to  manage  their  (typically 
constrained)  research  resources.  Unexpected  changes  on  the  part  of  the  observed  can  induce 
increases  in  the  above-mentioned  costs  which,  combined  with  increased  overhead  for  providing 
observational  coverage  itself  (e.g.,  transportation),  can  be  detrimental  to  observers'  control  over 
their  time,  equipment,  and  fiscal  budgets.  This  reversal  of  relative  control  suggests  itself  as  one 
explanation  for  the  frustration  occasionally  felt  by  some  CDTeam  personnel  during  the  course  of 
their  first  ethnographic  experience.  It  also  highlights  the  need  to  provide  for  the  unexpected  in 
planning  an  ethnographic  study,  particularly  if  total  observational  coverage  is  desired. 

Ethnographic  data  collection  can  vary  with  observer  capacities.  In  discussing  the  transcription 
ratio  earlier,  we  noted  some  variation  among  CDTeam  observers'  efficiency.  We  found  during  the 
course  of  this  exercise  that  there  were  also  variations  in  their  note-taking.  Specific  dimensions  of 


29 


consistent  variation  among  CDTeam  observers  included:  degree  of  detail;  attribution  of  comments 
to  speakers;  granularity  of  observations  (e.g.,  "blow-by-blow"  vs.  "summary  points");  volume  of 
note  documentation;  usage  /  misusage  of  relevant  references  (e.g.,  to  CCCD-specific  terminology); 
degree  of  observer  "interpretation"  from  actual  subject  utterances;  time  tracking;  and  subject 
attendance  tracking.  These  variations  were  largely  responsible  for  the  fact  (mentioned  earher)  that 
collaborating  observers  consistently  found  collaborative  transcription  slower  (and  more  fmstrating) 
than  individual  transcription. 

To  summarize  these  specific  points,  we  believe  that  ethnographic  studies  must  be  planned  with 
sufficient  attention  to  reahstic  time  and  resource  assessments.  Our  initial  ethnographic  exercise  did 
not  push  farther  on  to  review  and  analysis,  each  of  which  will  unavoidably  raise  the  researchers’ 
costs  for  this  type  of  enquiry.  Our  experience  is  that  the  conduct  of  such  studies  is  expensive  in 
terms  of  invested  time  and  incapable  of  precise  schedule  management.  We  suggest  that  future  such 
studies  could  address  these  problems  through  either  scaling  down  their  degree  of  observational 
coverage,  scaling  down  the  number  of  observers  employed,  or  delegating  much  of  the  direct 
observation  duties  to  lower-cost  personnel  (e.g.,  junior  staff  or  temporaries). 

We  believe  that  the  transcription  ratio  might  conceivably  be  reduced  in  practice  to  around  1:1, 
but  that  this  is  not  likely  unless  observers’  field  notes  are  more  directly  convertible  into  their 
archived  format.  One  obvious  solution  would  be  to  reduce  the  transcription  ratio  to  0:1  —  i.e., 
eliminate  the  need  for  post-session  transcriptions.  Specific  means  for  accomplishing  this  include 
the  use  of  portable  computers  or  personal  digital  assistants  (PDA’s  —  e.g.,  the  Apple  Newton™) 
by  observers  on-site.  Another  approach  which  might  prove  cost-effective  in  some  cases  would  be 
the  employment  of  proficient  professionals  drawn  from  other  work  settings  (e.g.,  stenographers). 

We  suggest  that  the  problems  of  situational  complexity  could  be  reasonably  addressed  through 
advance  familiarization  with  the  subject  scenario.  The  problem  of  inter-observer  variation  is  not  so 
easily  addressed.  One  could  attempt  to  address  this  by  employing  a  structured  protocol  for 
capturing  specific  events  or  content,  but  this  could  readily  degrade  the  richness  of  data  in  an 
ethnographic  approach  generally  and  disqualify  a  study  from  meeting  the  criteria  for  an 
ethnomethodological  approach  specifically.  Nonetheless,  such  a  "mixed-mode’’  tactic  might  well 
be  necessary  to  approximate  the  penultimate  ecological  validity  of  an  ethnographic  approach  within 
typical  operational  constraints.  Certainly  inter-observer  variation  could  be  partially  reduced 
through  effective  and  uniform  training,  but  we  doubt  this  is  a  guaranteed  cure.  The  most  effective 
solution  would  be  attention  to  selection  of  individual  observers.  In  our  (admittedly  hmited) 
experience,  the  main  sources  of  variation  were  frankly  personal  —  e.g.,  individual  skills  in 
listening,  notetaking,  transcription,  and  interpretation.  Within  our  (admittedly  small)  population  of 
observers,  relative  proficiency  in  these  sWlls  did  not  apparently  correlate  with  professional 
specialization  or  experience  in  other  modes  of  studying  human  behavior. 

Speaking  generally,  we  find  the  results  of  this  ethnographic  exercise  to  have  been  entirely 
consistent  with  the  issues  identified  by  Hughes  et  al.  (1994),  discussed  earlier.  Our  evidence  from 
burdened  schedules  and  transcription  ratios  demonstrates  the  "problems  of  time  pressure,"  at  least 
insofar  as  they  pertain  to  the  ethnographers’  own  work.  Even  though  the  CCCD  design  team  and 
Field  Demo  2  were  of  presumably  tractable  size,  our  assigned  resources  were  taxed  during  this 
study.  We  are  convinced  that  had  the  subject  population  been  larger  or  the  project  timeframe 
longer,  we  would  have  encountered  "problems  of  scale."  Finally,  our  experience  provides 
compelling  evidence  for  those  authors’  concern  with  "problems  of  the  ethnographer’s  role."  By 
this  we  mean  that  in  certain  instances  significant  energy  had  to  be  devoted  to  resolving  issues  of  the 
study’s  conduct  (as  opposed  to  the  study’s  content).  In  this  case,  such  issues  included: 

•  Permission  to  obtain  access  to  the  CCCD  Field  Demo  2  activities 

•  The  extent  and  frequency  of  CDTeam  observations 

•  The  types  of  activity  CDTeam  could  observe 
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•  The  "ground  rules"  for  CDTeam  observational  activity 

•  The  type(s)  of  data  collection  (e.g.,  audio  tapes)  which  could  be  employed 

•  The  type(s)  and  amount  of  background  documentation  to  be  delivered  to  CDTeam 

•  CDTeam  familiarization  with  the  background  (e.g.,  jargon,  tools)  to  the  subject  activity 

•  Scheduling  and  coordination  of  CDTeam  observation 

•  CDTeam  feedback  on  the  subject  team's  work 

•  Potential  CDTeam  contribution  to  the  Field  Demo  2  Validation  Test  Plan  documentation 

We  are  not  claiming  that  there  were  untoward  obstacles  in  addressing  these  issues,  nor  are  we 
inferring  that  our  relations  with  the  CCCD  project  and  its  personnel  were  anything  but  benign.  We 
are  simply  affirming  the  common  wisdom  in  ethnographic  and  qualitative  research  that  an 
unexpectedly  large  proportion  of  researchers'  over^  project  time  will  be  spent  on  such 
"procedural"  matters.  This  means  that:  (1)  significant  "ramp-up"  time  should  be  budgeted  for 
advance  negotiations  in  any  ethnographic  study;  (2)  good  relations  with  the  subjects  are  of  primary 
(and  not  peripheral)  importance;  and  (3)  researchers  should  budget  significant  time  resources  (with 
regard  to  procedure  matters)  above  and  beyond  those  required  for  the  observations  and  analyses 
themselves. 

The  foregoing  affirmation  of  the  burdens  entailed  in  ethnographic  research  does  not  mean  that 
we  believe  the  approach  to  be  something  best  avoided.  In  the  course  of  this  study,  we  repeatedly 
observed  interactions  involving  (e.g.)  negotiations  of  design  tradeoffs  which  set  the  context  for  the 
eventual  crewstation  redesign.  T^e  extent  to  which  such  tradeoffs  were  recognized  and  the 
grounds  upon  which  they  were  resolved  were  not  evident  in  the  final  crewstation  prototypes 
themselves.  We  developed  an  appreciation  for  the  extent  to  which  the  explanation  for  a  find 
product  lies  in  the  historicality  of  its  design  process  and  not  necessarily  in  deterministic  decision 
making  on  predictable  technical  or  functiond  issues.  This  observationd  study  affirmed  the 
criticality  of  design  rationde  as  a  factor  steering  the  course  of  design  and  as  a  form  of  data  which  is 
generally  useful  as  documentation  and  specific^y  vduable  in  direct  proportion  to  the  complexity 
of  the  design  project  and  its  product(s). 

Based  on  our  experience,  we  beUeve  that  unless  and  until  tools  for  red-time  accretion  of  design 
rationde  mature,  ethnograpMc  techniques  provide  a  viable  means  for  documenting  this  criticd 
background  data.  Consideration  of  an  ethnographic  approach  must  include  attention  to  the  diverse 
costs  well-documented  in  the  relevant  literature  and  affirmed  by  our  trid  experience.  Selection  and 
training  of  ethnographers  should  be  carefully  planned  and  carried  out.  Above  all,  planners  must 
recognize  the  relatively  "dl  or  nothing"  nature  of  such  research  work.  Because  ethnography's 
payoffs  are  predicated  on  the  comprehensivity  of  data  collected,  its  success  requires  a  commitment 
to  sustained  and  comprehensive  observation. 

ASC  Collaboration  Study 

In  discussing  Concurrent  Engineering  and  Integrated  Product  Design,  we  cited  the  work  with 
the  USAF's  Aeronauticd  Systems  Center  (ASC)  reported  in  McNeese  et  al.  (1993).  That  study 
elicited  knowledge  from  ASC  human  factors  specidists  concerning  (e.g.)  design  teams’ 
organization,  interpersond  relationships,  generalizable  experiences,  and  resolution  of  design 
problems.  Since  the  publication  of  the  1993  paper,  this  data  has  been  subsequently  andyzed  to 
identify  and  compile  tips,  clues,  and  issues  pertinent  to  how  information  technology  can  be  best 
configured  to  constructively  support  red-world  design  teams.  The  detailed  report  on  this  study  can 
be  found  in  Citera,  Selvaraj,  McNeese,  Brown,  &  Zaff.  (in  press).  The  primary  issues  identified 
are  summarized  in  the  following  points: 

1.  Design  rationale  is  important,  but  problematical.  The  ASC  experts  reported  that  past 
experiences  were  often  the  bases  for  criticd  design  decisions  and  the  justification  for  resolving 
tradeoffs,  even  foough  the  credibility  of  such  experientid  data  is  not  easily  assessed.  Persond 
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differences  in  experience  were  claimed  to  underlie  perceived  differentials  in  participants' 
credibility,  thus  affecting  the  social  dimension  of  multidisciplinary  design.  Design  rationale  data 
should  include  its  source,  but  this  entails  possible  problems  with  accountability  or  attribution. 

2.  Information  sharing  is  critical  in  multidisciplinary  design  teams.  The  ASC  experts  stated  that 
differences  in  jargon  and  conceptual  frameworks  are  very  problematical  for  multidisciplinary 
design  teams.  This  affimis  Boffs  (1987)  concern  for  such  teams  working  under  conditions 
equivalent  to  a  "Tower  of  Babel."  The  ASC  experts  suggested  that  effective  tools  for  compiling 
common  working  lexicons  and  indexing  shared  data  resources  are  much  needed.  This  result 
affirms  the  importance  of  a  common  medium  of  expression  ~  the  "shared  information  space" 
Robinson  (1991)  cites  as  one  of  CSCW's  most  critical  conceptual  contributions. 

3.  Joint  decision  criteria  are  critical  in  multidisciplinary  design  teams.  Data  analyses,  risk 
assessments,  and  the  like  are  complicated  enough  without  having  to  deal  with  multiple  disparate 
frames  of  reference.  Above  and  beyond  a  "shared  language"  for  depicting  design  tradeoffs, 
multidisciplinary  design  teams  must  arrive  at  a  consensus  regarding  the  justification  for  decisions 
on  those  tradeoffs.  This  result  affirms  the  problems  with  "different  ontologies"  in 
multidisciplinary  design  discussed  by  Bannon  and  Robinson  (1991),  and  it  cautions  against  the 
reification  of  simplistic  decisionm^ng  presumptions  in  groupware  artifacts,  such  as  those 
identified  for  group  decision  support  systems  (GDSS)  by  Whitaker  (1994). 

4.  Designers  need  lateral  access  to  experts  and  colleagues.  The  ASC  specialists  noted  the 
importance  of  obtaining  information  and  opinions  from  people  outside  the  multidisciplinary  design 
team  itself  -  e.g.,  from  technical  experts.  Means  for  supporting  this  function  include 
communications  systems  supporting  conferencing  (e.g.,  Usenet  news  groups)  and  repositories  for 
design  expertise.  This  affirms  the  need  for  good  communications  in  both  the  technics  and 
professional  senses. 

5.  Facilitating  productivity  and  effectiveness  of  design  meetings  is  a  priority.  In  accordance 
with  the  literature  on  concurrent  engineering,  the  ASC  experts  noted  that  multidisciplinary  design 
entails  many  meetings.  Tools  and  procedures  enhancing  meeting  focus,  agenda  formulation,  time 
usage,  and  distribution  of  outcomes  would  be  veiy  beneficial.  This  affirms  the  CSCW  emphasis 
on  tools  for  group  decision  support. 

To  summarize,  the  ASC  Collaboration  Study  produced  concrete  affirmation  for  many  of  the 
concerns  represented  in  CSCW  research  generally  and  the  CDTeam  agenda  specifically. 

CDT  Lab  Simulation  Studies 

During  our  research  planning,  we  identified  a  third  class  of  research  activities  which  did  not 
readily  fall  under  either  the  categories  of  experimental  or  observational  studies  as  described  earlier. 
These  activities  were  those  which  derived  from  that  element  of  the  August  1993  mission  statement 
mandating  us  to  "...model  and  simulate  advanced  groupware  to  assess  human  and  technology 
demands  and  implementation  feasibilities."  This  entailed  setting  up  and  conducting  interaction 
between  users  and  available  collaborative  technologies,  toward  evduating  the  human  and  other 
operant  factors  at  work.  We  termed  such  activities  simulation  studies,  based  on  the  idea  of  their 
entailing  "simulations"  of  workplace  activity  supported  by  innovative  technologies.  With  respect 
to  the  CDT  Lab  (and  its  equipment)  as  a  reconfigurable  testbed,  we  envisioned  simulation  studies 
to  proceed  recursively  as  follows: 

•  Study  design  teams  in  the  Group  Interface  Facility  testbed 

•  Analyze  design  teams’  usage  of  the  Group  Interface  technologies 

•  Apply  the  results  of  such  analyses  to  evolve  the  Group  Interface  Facility 
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Such  simulation  studies  were  most  commonly  a  test  of  one  or  another  feature  within  the 
framework  of  a  real-world  group  activity.  Many  of  the  CDTeam's  simulation  studies  were 
conducted  in  conjunction  with  the  provision  of  knowledge  elicitation  and  facilitation  services  to 
local  (USAF  and  associated  contractor)  clients.  These  services  typically  relied  on  the  concept 
mapping  techniques  developed  in  the  earlier  AKADAM  work  (McNeese  et  ai,  in  press). 
CDTeam's  facility  in  such  activities  allowed  us  the  ability  to  study  clients'  behavior  in  the  course  of 
activities  involving  the  use  of  IT  (e.g.,  software,  large  display  devices).  Such  work  was  aimed  at 
identifying  human  factors  issues  relevant  to  group  IT  usage.  One  such  issue  we  studied  was  the 
interaction  between  collaborators  and  their  shared  information  spaces.  In  the  context  of  CDTeam- 
faciUtated  sessions,  this  interaction  was  between  clients  and  the  representations  they  jointly 
constructed. 

Depictional  Lock-On:  An  Instance  of  Flipover 

Access  to  a  common  medium  for  collective  activity  is  one  of  the  fundamental  elements  in 
CSCW.  Bannon  and  Schmidt  (1989,  p.  364)  identify  "sharing  an  information  space"  as  a  "core 
issue  for  CSCW,"  De  Ivlichelis  (1990)  cites  "information  sharing"  as  the  key  support  for 
collaborative  activity,  anc  Robinson  (1991)  cites  shared  information  space  as  one  of  the  most 
important  "CSCW  specific  concepts."  Both  Robinson  (1991)  and  Bannon  (1991b)  credit 
Thompson  (1984)  with  this  concept,  which  is  implicit  in  Engelbart's  seminal  visions  of  shared  IT 
applications  (cf.  1963;  1982)  and  (termed  shared  environment)  Elhs  et  a/.'s  (1991,  p.  40) 
definition  for  "groupware."  This  section  will  describe  an  enquiry  into  human  factors  issues 
surrounding  team  members'  relative  degree  of  attention  to  their  shared  information  spaces  (as 
opposed  to  each  other)  during  collaboration.  This  topical  focus  is  best  framed  with  regard  to  two 
theoretical  models  from  the  CSCW  literature. 

Robinson  (1989;  1991)  differentiates  between  two  modes  of  group  interactivity  which  he 
characterizes  as  "levels  of  language."  The  formal  level  consists  of  structured  modes  of 
interactivity,  usually  guided  by  the  model  or  rules  embodied  in  (e.g.)  a  software  application.  This 
level  "...is  essential  as  it  provides  a  common  reference  point  for  participants  ...  a  sort  of  'external 
world'  that  can  be  pointed  at,  and  whose  behavior  is  rule  governed  and  predictable"  (1989,  p.  56). 
The  cultural  level  denotes  "...a  language  that  is  actually  spoken  by  a  community  of  people." 
Conversations  at  the  cultural  level  may  involve  "...understanding,  interpreting,  and  changing 
'items'  at  the  formal  level... ,"  "...procedural  and  annotative  activities...  ,"  and  "...also  any  other 
social  or  interpersonal  aspects  (relevant  to  the  'problem'  or  not)  that  the  participants  wished  to 
introduce."  (Ibid.). 

In  a  similar  vein,  Whitaker  (1992)  outlines  a  venue  framework  for  analyzing  IT-supported 
collaboration,  employing  the  legal  sense  of  "venue"  (jurisdiction)  to  connote  a  setting  or 
circumscriptive  medium  l.>r  action.  The  framework  derives  from  Maturana  and  Varela's  (1980) 
concept  of  phenomenological  domain  —  a  realm  of  action  defined  by  the  individual  properties  of  its 
constituents  and  their  collective  transformations  or  interactions.  Accordingly,  electing  one  of  the 
venues  focuses  attention  on  the  entities  and  actions  constituting  it,  thus  specifying  a  vantage  point 
for  analysis.  Two  of  these  venues  will  be  employed  for  the  purposes  of  this  discussion.  The 
depictive  venue  is  the  "informational"  context  —  the  composite  set  of  all  symbohzable  data 

available  to  the  decision  maker.  As  a  target,  it  is  the  composite  set  of  all  symbohzable  data 

generated  during  the  decision  making  process.  The  technological  interventions  effected  by 

groupware  (e.g.,  shared  data  capacities)  are  typically  realized  in  this  depictive  venue.  The 

discursive  venue  is  the  communicational  setting  subsuming  all  direct  interactivity  in  a  meeting,  i.e., 
the  overt  activity  linking  participants  to  each  other.  The  interactional  interventions  effected  by 
structured  procedures  are  primarily  realized  in  this  discursive  venue. 

These  two  taxonomic  models  address  similar  issues,  but  they  are  not  necessarily  isomorphic. 
Robinson's  distinction  of  formal  versus  cultural  level  language  hinges  on  the  degree  of 
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structuration  in  communicative  activity.  Whitaker's  distinction  of  depictive  versus  discursive 
venues  hinges  on  a  differentiation  of  settings  or  media  in  which  communicative  activity  is 
conducted.  We  will  need  to  invoke  both  models  in  delineating  the  relative  interplay  of  natural 
conversation  (cultural  level  language  /  discursive  venue)  and  structured  representational  schemata 
(formal  level  language  /  depictive  venue)  in  collaborative  IT  usage. 

Our  interest  lay  in  exploring  the  operational  confluence  of  the  depictive  and  discursive  venues, 
to  the  (typically  substantial)  extent  that  participants  may  address  each  other  (discursive  venue)  only 
through  the  mediate  textual  data  space  (depictive  venue).  More  specifically,  we  wished  to  explore 
what  Robinson  and  Bannon  (1991)  term  flipover  —  the  phenomenon  in  which  "...the  object  of  an 
interpretation  and  the  interpretation  itself  change  places."  (p.  225,  emphasis  in  the  original).  They 
provide  multiple  examples  characterizing  flipover  in  terms  of  a  mutually  created  interpretive  unity 
(e.g.,  a  position,  a  document)  becoming  a  fixed  object  subject  to  later  interpretation.  Flipover  is 
thus  characterized  as  an  unavoidable  phenomenon  in  interaction  (therefore  in  group  work).  On  the 
other  hand,  flipover,  as  a  "...dialectical  movement  between  an  object  of  interpretation  and 
interpretation  itself..."  (Bannon  &  Robinson,  1991,  p.  223)  imparts  utility  and  power  to 
collaboration. 
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Figure  4.  Model  for  problem  solving 

Now  let  us  illustrate  the  relevance  of  the  concepts  of  flipover,  formal  /  cultural  level  languages, 
and  discursive  /  depictive  venues  in  framing  human  factors  issues  of  IT-supported  design 
collaboration.  Whitaker's  (1992;  1994)  model  for  problem  solving  (illustrated  in  Figure  4)  applies 
to  design  decision  making  generally.  In  the  course  of  taking  steps  toward  problem  resolution 
(steps  1-6  in  the  figure),  problem  solvers  typically  shift  from  consideration  of  their  real  world 
issues  to  one  or  more  representations  (e.g.,  rough  sketches).  Once  those  representations  have 
been  manipulated  to  the  point  of  framing  a  solution  (e.g.,  a  finished  design  drawing),  the  focus 
shifts  back  to  the  real  world  to  effect  implementation.  The  first  transition  (cf.  Step  2  in  the  figure) 
corresponds  to  a  flipover,  a  shift  from  cultural  to  formal  level  language,  and  a  shift  of  emphasis  to 
the  depictive  venue.  This  point  is  the  critical  locus  at  which  insufficiencies  in  problem  recognition 
become  reified  in  working  models  and  the  attendant  costs  of  subsequent  correction  grow 
dramatically. 

Observations  of  CDTeam  client  interactional  behavior  within  the  structured  setting  of  our 
facihtated  services  revealed  that  shifts  of  relative  focus  between  the  natural  conversational  space 
and  the  emerging  structured  representations  (e.g.,  concept  maps,  graphs,  lists)  did  in  fact  occur. 
For  example,  it  was  common  for  clients  to  initiate  a  knowledge  elicitation  session  by  conversing 
with  their  fellows  or  with  the  facilitator(s)  without  significant  conversation  being  directed  toward 
the  tools  or  representational  devices  themselves.  During  the  course  of  many  such  sessions,  we 
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noticed  a  progressive  shift  in  conversational  indexicality  (direetion  or  focus  of  discursive  reference) 
from  the  task  or  issue  under  discussion  per  se  to  the  emerging  representational  product.  In  other 
words,  as  meetings  went  on,  we  noted  that  participants  talked  less  in  terms  of  the  subject  matter 
and  more  in  terms  of  the  emergent  representations  of  that  subject  matter.  In  some  cases,  this  shift 
was  so  pronounced  that  clients  were  framing  their  comments  almost  wholly  in  terms  of 
components  of  the  representational  schemata  (as  opposed  to  features  of  the  issue  under 
discussion). 

Our  earliest  observation  of  this  shift  occurred  in  July  1993,  during  a  concept  mapping  session 
with  a  single  subject  matter  expert  (SME)  in  the  AL/CFH  MultiMedia  Room  —  a  eomputer- 
supported  presentation  and  meeting  facility.  The  primary  concept  mapper  employed  Inspiration™ 
software  on  an  Apple  Macintosh™  computer  as  the  mapping  medium.  The  SME  observed  the 
emerging  representation  on  a  large  rear-projection  video  display  at  the  front  of  the  room.  As  the 
SME  talked  about  the  subject  of  interest  (plans  for  a  joint  services  equipment  evaluation  program) 
the  mapper,  using  keyboard  and  mouse,  generated  nodes  and  links  corresponding  to  die  major 
points.  Five  other  CDTeam  members  were  seated  to  the  side,  and  they  only  oeeasionally  entered 
into  conversation  with  the  SME  and  /  or  the  mapper.  One  of  these  CDTeam  members  observed  the 
interaction  between  the  SME  and  the  others,  noting  any  instance  where  the  SME  spoke  with 
reference  to  the  concept  mapping  rather  than  with  strict  regard  to  the  subject  matter  (e.g,  the 
equipment  of  interest  or  the  proposed  evaluation  effort).  The  data  collection  method  was 
handwritten  notes,  which  were  then  transcribed  into  an  electronic  file  and  categorized  topieally. 

To  use  Robinson's  terminology,  the  formal  level  language  consisted  of  the  node-and-link 
concept  mapping  representation.  The  SME  was  advised  prior  to  the  meeting  that  he  would  control 
the  session's  discourse,  so  the  conversation  between  the  SME  and  the  mapper  was  not  "formal 
level"  -  at  least  not  initially.  The  cultural  level  language  was  simply  the  natural  conversation  of  the 
SME  and  the  mapper.  To  use  Whitaker's  terminology,  the  diseursive  venue  was  comprised  of  the 
meeting  room  spaee,  the  participants,  natural  language  and  the  other  behaviors  (e.g.,  gestures) 
comprising  the  interpersonal  communication  between  SME  and  mapper.  The  depictive  venue 
consisted  of  the  computer,  its  control  interface,  and  the  large  eommon  display  onto  whieh  the 
concept  map  was  projected. 

During  the  two  hours  and  forty-five  minutes  of  this  1993  session,  the  SME  progressively 
shifted  from  talking  about  his  program  planning  to  directing  the  construction  and  reconfiguration  of 
the  concept  map  on  the  video  display.  Phrased  another  way,  his  primary  style  of  interaction  with 
the  mapper  shifted  from  dietation  (speaking  for  the  record)  to  direction  (telling  how  the  reeord  itself 
should  be  modified).  During  the  first  10  minutes  of  the  session,  he  sat  eomfortably  baek  in  his 
chair  and  directed  his  primary  gaze  at  the  mapper.  By  the  time  the  first  quarter  hour  was  complete, 
he  had  come  forward  in  his  chair,  leaned  on  the  table  before  him,  and  directed  his  primary  gaze  at 
the  video  display.  Whenever  engaged  in  the  concept  mapping  directly  (as  contrasted  with  side 
conversations),  the  SME  consistently  adopted  this  latter  postural  orientation  throughout  the 
remainder  of  the  session. 

A  second  shift  occurred  during  this  same  period  (10-15  minutes  into  the  session).  The  SME 
began  speaking  with  regard  to  the  display  and  the  eoncept  map  in  his  interactions  with  the  mapper. 
Some  aspects  of  the  display-directedness  were  clearly  related  to  the  constrained  physical 
affordances  of  the  setting  (e.g.,  hmited  display  area).  Examples  of  this  included  (1)  directing  the 
mapper  to  scroll  the  display  image  to  bring  a  specific  portion  of  the  map  into  view  and  (2) 
commenting  to  the  mapp-ir  on  the  display's  size,  clarity,  and  legibility.  These  references  to  the 
display  were  understandable  responses  to  the  circumstances  within  which  the  knowledge  elicitation 
was  occurring.  More  interestingly,  the  SME  responded  in  a  similar  fashion  with  respect  to  the 
knowledge  elicitation  process  itself.  By  this  we  mean  that  the  SME  progressively  framed  his 
conduct  as  an  interviewee  with  reference  to  the  representational  device  (the  concept  map)  projected 
on  the  large  display.  Examples  of  this  representation-directedness  ineluded: 
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•  Directing  the  mapper  to  add  one  or  more  nodes  extending  from  a  specific  point  in  the  map 

(e.g.,  "Add  a  node  off  that  one  there.") 

•  Directing  the  mapper  to  delete  nodes  (usually  empty  ones  left  over  from  some  previous 

manipulation) 

•  Asking  to  review  some  portion  or  the  entirety  of  the  map  before  continuing 

•  Directing  the  mapper  to  modify  location  of  a  node 

•  Directing  the  mapper  to  cluster  or  re-arrange  sets  of  nodes 

•  Directing  the  mapper  to  ehange  the  entire  text  label  for  a  node  or  for  a  relational  link  between 

nodes 

•  Directing  the  mapper  to  add  explanatory  text  or  terms  to  node  or  link  labels 

•  Directing  the  mapper  on  color  coding  certain  of  the  nodes 

•  Directing  the  mapper  to  ehange  or  correct  spelling  in  the  text  labels  for  nodes  or  links 

We  found  the  same  general  effect  in  sessions  involving  multiple  SME's  or  clients.  In  Mareh 
1994,  CDTeam  faeilitated  two  planning  sessions  with  a  group  of  six  government  managers  from  a 
USAF  research  laboratory  ,  each  of  two  hours'  duration.  The  goal  of  the  meetings  was  to  outline 
specific  requirements  for  a  large-scale  systems  integration  project.  The  site  was  the  CDT  Lab 
meeting  space,  and  the  knowledge  elicitation  method  was  again  eoneept  mapping.  The  SME's 
were  seated  side-by-side  (as  a  panel)  before  a  wall  of  whiteboards,  on  wWch  the  single  knowledge 
elicitor  constructed  a  concept  map.  In  parallel,  another  CDTeam  member  was  transcribing  the 
emerging  map  into  an  electronic  version  projected  onto  a  six-foot-square  screen  located  to  the 
SME's  left  at  a  45  degree  angle.  The  actual  point  of  this  parallel  activity  was  immediate 
transeription  of  the  whiteboard  map  into  electronie  format  for  ready  distribution  at  the  session's 
end.  The  transcription  copy  was  projected  during  the  session  as  a  eonvenience,  providing 
additional  display  feedback  to  the  SME's.  Both  sessions  were  videotaped  from  two  eamera  angles 
(one  facing  the  panel,  one  looking  over  the  back  of  the  panel  toward  the  mapper  and  the 
whiteboards).  CDTeam  review  of  these  videotapes  was  the  method  of  analysis  with  respect  to  this 
particular  issue. 

As  with  the  individual  ease  described  above,  the  SME  panel  members  exhibited  progressive 
shifts  in  postural  orientation  and  gaze  directedness  with  respect  to  the  concept  map  being 
eonstructed.  The  point  of  noticeable  shift  occurred  about  10  or  15  minutes  into  the  session.  The 
consisteney  and  conclusivity  of  this  shift  was  not  so  pronouneed  as  for  the  individual  ease, 
apparently  owing  to  occasional  general  erosstalk  among  the  panel  members  and  more  specific 
rounds  of  crosstalk  initiated  in  response  to  some  particular  point  or  question  during  the  session. 
Similarly,  these  panel  members  exhibited  the  general  shift  "from  dietation  to  direction"  noted  in  the 
individual  ease.  When  making  an  initial  point,  each  panel  member  progressively  came  to  direct  the 
course  of  the  concept  mapping  rather  than  simply  dietate  a  proposition.  As  for  the  orientational 
shift,  the  consisteney  of  this  conversational  shift  was  less  than  in  the  individual  ease.  As  for  the 
orientational  shift,  erosstalk  (e.g.,  responding  to  a  suggested  point,  adding  items  to  a  list  initiated 
by  someone  else)  was  the  apparent  correlate  to  falling  back  into  "dictation  mode"  with  respect  to 
the  concept  mapper. 

In  terms  of  speeifie  instances  where  either  prior  propositions,  novel  referents  (e.g.,  names, 
acronyms),  or  their  representations  beeame  points  of  subsequent  reference,  the  examples 
demonstrate  the  flipover  phenomenon.  Flipover  pertains  to  the  meaning  of  an  emerging  group 
produet,  as  evideneed  by  Bannon  and  Robinson's  (1991)  allusion  to  the  statement  in  Neuwirth  et 
al.  (1990,  p.  185)  that  "...the  partially  completed  produet  plays  an  important  role  in  this  process; 
The  partially  eompleted  produet  becomes  part  of  the  task  environment  and  constrains  the 
subsequent  eourse  of  the  design..."  In  terms  of  Robinson's  double  level  language,  flipover  occurs 
when  cultural  level  language  feeds  forward  to  effect  the  regularization  of  a  formal  level  language. 
Consistent  with  its  definition  in  the  literature,  flipover  is  a  dieoretieal  model  for  an  epistemologieal 
phenomenon  discernible  in  terms  of  cognitive  constructs  (i.e.,  yet  more  theoretical  models). 
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In  contrast,  the  behavioral  shifts  we  observed  in  attentional  orientation  and  behavioral  framing 
with  respect  to  the  facilitator  /  mapper  concern  subjects'  orientation  to  the  depiction  of  the  emergent 
product  in  their  shared  information  space.  In  other  words,  we  observed  shifts  with  respect  to  the 
"vehicle,"  as  well  as  the  "content,"  of  the  emergent  information  artifact  (e.g.,  a  concept  map).  To 
distinguish  this  effect  from  flipover,  we  term  it  depictional  lock-on.  In  terms  of  Whitaker's  venue 
framework,  depictional  lock-on  marks  the  point  at  which  communicative  activity  within  the 
discursive  venue  becomes  demonstrably  coupled  to  manipulative  conduct  within  the  depictive 
venue.  Depictional  lock-on  is  a  theoretical  model  for  a  behavioral  phenomenon  discernible  in  terms 
of  concrete,  potentially  quantifiable,  evidence  (e.g.,  gaze  directedness,  conversational  phrasing)  of 
the  sort  amenable  to  human  factors  research. 

Even  though  our  observations  of  the  depictional  lock-on  phenomenon  were  secondap^  to  the 
sessions'  goals  and  the  data  collection  was  ad  hoc,  there  is  consistency  in  the  Umited  evidence  to 
date.  Whereas  flipover  is  described  as  a  continuously  recursive  process,  depictional  lock-on 
operates  as  a  transition  across  a  delineable  boundary  in  the  course  of  a  collaborative  session  (e.g., 
10-15  minutes  into  the  example  sessions  above).  In  the  example  sessions  described  above, 
depictional  lock-on  typically  dissipated  or  broke  down  when  the  subject(s)  had  to  interact  with 
others  in  the  room  —  i.e.,  when  communicative  activity  within  the  discursive  venue  decoupled 
from  the  depictive  venue.  In  the  case  of  the  group  sessions,  participants  exhibited  differential 
degrees  of  lock-on  generally  and  during  crosstalk.  A  striking  example  occurred  in  the  first  group 
session.  One  participant  consistently  faced  a  second  addressed  during  crosstalk,  but  the  addressee 
consistently  responded  verbally  while  gazing  at  the  concept  map  -  even  punctuating  response 
comments  with  pointing  gestures  toward  the  whiteboards.  The  lock-on  effect,  after  its  initial 
appearance,  could  be  re-established  after  a  break  with  no  discernible  delay  corresponding  to  the 
initiation  period. 

More  suggestively,  it  was  observed  that  the  subject(s)  in  the  example  sessions  above  tended  to 
phrase  their  statements  regarding  basic  sufficiency  and  finality  for  the  session  in  terms  of  the 
concept  map  representation  rather  than  their  perceived  coverage  of  the  subject  matter.  In  the 
individual  case,  the  SME  stated  "Now  I  have  a  concept  map"  at  about  1  hour  into  the  session,  and 
the  close  of  the  session  came  when  he  (1)  suggested  the  map  was  approaching  comprehensivity 
(without  reference  to  the  subject  matter)  and  then  (2)  stated  that  the  map's  evolution  had  reached  a 
good  stopping  point.  At  the  end  of  one  of  the  group  sessions,  when  the  SME's  were  trying  to 
specily  the  next  step  in  their  planning  process,  they  discussed  (1)  taking  the  generated  concept 
maps  and  showing  them  to  higher  management  and  (2)  framed  higher  management's  prospective 
feedback  in  terms  of  reaction  to  the  concept  maps  themselves. 

Production  of  a  shared  informational  artifact  is  a  focal  scenario  for  CSCW  research  and  the 
explicit  goal  of  knowledge  elicitation  sessions  such  as  the  examples  above.  Sharing  a  medium  of 
expression  is  one  of  the  benefits  claimed  for  groupware  generally  and  the  earlier  AIC\DAM  work 
specifically,  predicated  on  the  idea  that  a  common  mode  of  expression  or  depiction  would 
counteract  the  "Tower  of  Babel"  effect  which  bedevils  multidisciplinary  design  teams  (Boff, 
1987).  Identification  of  this  problem  and  the  suggestion  of  a  shared  medium  as  its  solution  are 
substantiated  by  the  findings  of  the  ASC  Collaboration  Study  reported  elsewhere  in  this  report  and 
in  Citera,  Selvaraj,  McNeese,  Brown,  and  Zaff  (in  press). 

Before  letting  the  issue  rest  with  prescription  of  shared  media,  however,  one  must  ask  if  any 
such  "shared  language"  is  necessarily  an  improvement  and,  if  so,  on  what  terms.  Given  the  scope 
of  this  discussion,  we  shall  address  this  in  terms  of  the  shared  medium's  influence  over  the  course 
of  constructive  (e.g.,  text-producing)  collaboration.  Earlier,  we  noted  flipover  being  defined  when 
a  "...  partially  completed  product  becomes  part  of  the  task  environment  and  constrains  the 
subsequent  course  of  the  design..."  (Neuwirth  et  ah,  1990,  p.  185,  emphasis  added).  This  is 
affirmed  by  empirical  evidence  that  collaborative  affordances  are  critical  to  usability  of  all  shared 
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media  and  adoption  of  novel  ones  (Luff,  Heath,  &  Greatbatch,  1992).  Such  ongoing  constraint 
can  be  construed  either  as  positive  (progressive  focus)  or  as  negative  (progressive  tunnel  vision). 
Let  us  consider  both  sides  of  this  tradeoff  in  turn. 

The  positive  payoffs  of  shared  information  have  been  accepted  without  question  since  the  days 
of  Engelbart's  seminal  work.  Attention  to  the  object  of  mutual  construction  is  presumed  to 
facUitate  common  understanding,  provide  a  basis  for  error-checking,  stimulate  points  for  further 
development,  and  delineate  the  grounds  for  consensus  building.  We  had  in  fact  noted  the 
occurrence  of  such  effects  in  group  concept  mapping  sessions  (Zaff,  Hughes,  McNeese,  Brown, 
&  Citera,  1993).  Focused  attention  on  primary  subject  matter  is  a  key  aspect  of  that  style  of 
effective  creativity  termed /Zow  (Csikszentmihalyi,  1990)  —  the  very  sort  of  constructive  activity 
we  wish  to  facilitate  in  design.  Illustrating  the  potentially  negative  side  of  flipover  /  depiction^ 
lock-on  requires  no  more  than  extending  the  speech  metaphor  underlying  the  "Babel"  problem 
statement  and  the  prescribed  "shared  language"  cure.  This  unavoidably  brings  one  to  the  historical 
examples  of  shared  language  in  collaboration  —  the  pidgin  dialects  generated  to  facilitate  trade. 
Pidgin  dialects  are  characterized  by  a  hmited  lexicon  and  an  extremely  simplified  grammatical 
structure.  Bannon  and  Robinson  (1991)  defined  flipover  in  the  course  of  discussing  the  dangers 
of  over-reliance  upon  models  and  representations  in  the  design  process.  Our  observations  of 
flipover  and  depictional  lock-on  give  us  reason  to  question  if  there  are  tradeoffs  to  be  considered 
between  the  benefits  of  shared  media  and  the  dangers  of  describing  the  complexities  of  large 
systems  using  the  equivalent  of  a  pidgin  dialect. 

The  epistemological  constraints  of  structured  knowledge  engineering  schemata  (including  the 
semantic  network  model  exemplified  in  concept  mapping)  were  among  the  problems  participatory 
practices  such  as  AKADAM  were  created  to  alleviate  (Zaff,  McNeese,  &  Snyder,  1993). 
Achieving  better  design  through  collaborative  technology  entails,  but  may  not  stop  with,  making 
Collaborative  Design  practices  user-centered.  Our  parallel  interest  in  Collaborative  Technology  has 
led  us  to  explore  how  shared  information  spaces  (the  depictive  venue)  interact  with  and  affect  the 
constructive  process  we  call  design.  Our  application  of  current  CSCW  theory  in  observing  real 
collaborative  tasks  has  given  us  clues  to  at  least  one  human  factors  phenomenon  (depictional  lock- 
on)  whose  further  study  may  inform  effective  shared  expression. 

CDT  Lab  Technology  Studies 


Groupware  Evaluations 

The  basic  CDT  Lab  IT  infrastructure  (described  elsewhere  in  this  report)  provided  us  with 
some  of  the  basic  capacities  for  collaboration  —  e.g.,  networking,  email,  and  file  sharing.  Because 
AL/CFH  migrated  from  a  centralized  to  a  distributed  mode  of  internal  networking  during  this 
reporting  period,  we  also  drew  on  our  daily  experiences  with  the  Novell  NetWare™-based  LAN 
and  WordPerfect  Office™  (now  marketed  as  Novell  GroupWise™).  In  addition,  we  evaluated  a 
variety  of  available  software  categorizable  as  groupware,  collaboration  support  tools,  and/or  tools 
relevant  to  interfacing  teams  with  computers.  These  software  packages  included: 

•  NCSA  Telnet  (TCP/IP  communication  software  allowing  desktop  access  to  the  Internet) 

•  NCSA  Mosaic  (HTML  browser  for  the  World  Wide  Web) 

•  NetScape  (HTML  browser  for  the  World  Wide  Web) 

•  TurboGopher  (Macintosh  software  enabling  direct  Gopher  information  service  access) 

•  Aspects™  (Shared  medium  for  textual  and  graphic  manipulation) 

•  Co-Motion™  (Shared  structured  conferencing  tool) 

•  Share  Vision™  (Desktop  video  conferencing  package  for  the  Apple  Macintosh™) 

•  MacHandwriter™  (Pen-based  handwriting  recognition  package  for  the  Apple  Macintosh™) 

•  ShrEdit  (Shared  text  editor  software) 

•  Timbuktu  (Software  for  sharing  screen  and  control  capacities  between  two  computers) 
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•  MacShapa™  (Softwrre  for  exploratory  sequential  data  analysis) 

•  TAKE  (Concept  mapping  software  for  knowledge  elicitation) 

•  Inspiration™  (Graphic  software  for  knowledge  elicitation  and  structuring) 

•  Collage  (Collaborative  data  visualization) 

•  Euclid  (Conferencing  support) 

•  Mac  Meeting  Manager  (Conferencing  support) 

•  MacDiscuss  (Conferencing  support) 

•  Maven  (Digital  voice  conferencing  support) 

•  Meeting  Maker  (Conferencing  support) 

•  Oval  (Information  management) 

•  PREP  (Collaborative  editing  support) 

•  Town  Meeting  (Conferencing  support) 

•  TalkShow™  (Screen  sharing  application  for  PCs  running  Windows™) 

Of  these  various  packages,  we  consistently  found  that  a  combination  of  basic  functionality 
(e.g.,  text  editing)  combined  with  rehability  was  most  useful.  By  "basic  functionality,"  we  mean 
that  the  software  provided  support  for  common  (and  hence  very  generalizable)  functions.  The  ^st 
example  of  such  comprehensive  support  is  Aspects™,  which  allows  people  using  multiple 
Macintoshes  over  a  LAN  to  share  screens  and  simultaneously  manipulate  document  files  using  text 
editing,  bitmap  painting,  or  object-oriented  drawing  tools  completely  consistent  with  the  Macintosh 
platform's  (and  interface’s)  relevant  global  characteristics.  In  other  words,  anyone  familiar  with 
the  most  basic  Macintosh  interface  conventions  for  these  functions  already  knew  the  affordances  of 
the  Aspects  package. 

The  unexpected  "down  side"  of  this  transparency  was  that  such  readily-usable  tools  were  seen 
as  somewhat  simplistic  cr  trivial  once  the  user  realized  the  correspondences  between  them  and 
ordinary  Macintosh  capacities.  There  is  a  general  difficulty  in  recognizing  and  appreciating  the 
fundamental  motivation  for  CSCW  research  -  the  collective  work  team  and  the  collective  IT 
infrastmcture  are  the  foci  of  interest  (as  opposed  to  the  individual  worker  and  the  single  IT 
artifact).  We  believe  this  "blindness"  carries  over  into  the  assessment  of  groupware.  In  the  case  of 
Aspects™,  the  value  added  by  synchronous  screen  sharing  did  not  seem  to  register  on  test 
evaluators,  who  consistently  assayed  the  package's  utility  in  terms  of  its  utility  at  their  respective 
workstations.  In  effect,  they  eventually  downgraded  their  impressions  of  Aspects™,  apparently 
on  the  basis  of  seeing  it  as  a  simple  combination  of  functions  found  elsewhere  (e.g.,  in 
SimpleText,  MacPaint,  and  MacDraw).  CDTeam,  on  the  other  hand,  continued  to  upgrade  their 
impression  of  Aspects,  based  on  their  ability  to  rehably  employ  it  in  a  wide  variety  of  situations 
and  over  significant  lengths  of  time.  In  other  words,  the  addition  of  collaborative  affordances,  no 
matter  how  elegantly  done,  was  not  consistently  recognized  as  a  significant  achievement. 

The  more  specialized  an  application  (e.g.,  the  more  it  relied  on  a  specific  data  representation  or 
procedure),  the  less  useful  we  found  it.  TMs  derived  in  part  from  the  common  inability  to  import 
and  export  file  materials.  It  also  derived  from  the  necessity  of  overcoming  a  significant  "learning 
curve"  before  the  product's  utility  was  either  obtained  or  appreciated.  On  the  other  hand,  such 
specialized  packages  carried  an  aura  of  novelty  about  them  which  (at  least  initially)  aroused  user 
interest  and  motivated  the  learning  process.  A  good  example  of  this  was  the  Co-Motion™ 
conference  support  software,  of  which  we  evaluated  version  1.5  in  autumn  1994.  Co-Motion  is  a 
collaborative  tool  for  organizing  and  collating  textual  comments  on  a  given  issue  or  topic.  It 
provides  a  reasonably  intuitive  graphical  user  interface  through  which  an  "essentially  unlimited" 
number  of  users  enter  tlieir  comments,  etc.,  over  an  AppleTalk™  or  AppleTalk™-compatible 
network.  Remote  users  n-ay  access  Co-Motion  sessions  via  Apple  Remote  Access™  software  pd 
a  modem.  It  also  offers  users  the  abiUty  to  express  preferences  or  rankings  on  selected  points 
related  to  the  issues  under  discussion.  Finally,  summary  reports  of  the  session(s)  can  be  printed 
out. 
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Any  number  of  Co-Motion  users  may  link  into  a  session  resident  on  a  host’s  machine.  Each 
such  user  is  confronted  with  a  graphical  Macintosh  window  with  a  tool  palette.  There  are  4  tools: 

•  A  Browser  Tool  (a  pointing  hand) 

•  A  Target  Tool  (denoting  Goals  or  Objectives) 

•  A  Shovel  Tool  (denoting  Problems  or  Issues  to  be  worked  through) 

•  An  Action  Tool  (denoting  Action  Items) 

Each  user  clicks  on  the  appropriate  tool,  then  moves  his/her  cursor  to  the  main  window. 
Clicking  the  Browser  Tool  on  a  given  spot  in  the  main  window  selects  an  existing  item.  Clicking 
one  of  the  other  tools  on  a  given  spot  establishes  a  new  (blank)  item  of  that  type.  Each  such  item 
has  a  text  box  allowing  up  to  three  short  lines  of  text.  Double-clicking  on  an  extant  item  brings  up 
an  Info  Window  for  the  given  item  (or  Idea).  Within  the  Info  Window,  the  user  may: 

•  Enter  text  annotations  in  a  scrollable  window 

•  Search  through  the  set  of  other  extant  items  of  the  same  type  as  the  one  selected 

•  Designate  responses  or  opinions  on  the  pre-packaged  questions  /  issues  associated  with 

each  type  of  idea  (Polls) 

Each  of  these  operations  are  done  under  conditions  of  anonymity  —  i.e.,  no  one  can  tell  which 
user  posts  which  Idea,  which  notes,  etc.  A  separate  Chat  Window  is  provided  for  posting 
messages  among  participants  (i.e.,  notes  which  are  not  meant  to  be  inserted  into  the  session  record 
itself).  This  Chat  Window  is  common  to  all  users,  and  (unlike  some  other  groupware  products)  it 
is  not  possible  for  one  user  to  direct  a  message  to  another  specific  user  individually.  Co-Motion 
can  be  used  synchronously  or  asynchronously.  This  affords  flexibility  to  lay  out  an  issue  in  a 
session,  then  allow  others  to  participate  as  they  are  able.  We  found  Co-Motion  to  be  an 
inexpensive  tool  best  suited  for  accreting  and  annotating  a  problem  overview  and  initial  opinions. 
It  would  be  best  suited  for  policy  or  decision  making  conferencing  among  (e.g.)  managers  on  a 
local  area  network. 

On  the  other  hand,  we  found  Co-Motion  to  be  a  specialized  product  suited  only  for  issue 
discussion.  Because  it  does  not  provide  screen  or  application  sharing,  it  is  not  suited  for 
synchronous  collaborations  of  the  sort  supported  in  Aspects.  Because  it  does  not  provide  an 
internal  capacity  for  users  to  direct  messages  to  each  other  individually,  it  is  not  suited  for  general 
messaging.  It  is  not  suited  to  applications  requiring  collaborative  action  (e.g.,  group  editing),  nor 
is  it  well-suited  for  final  decision  making  (e.g.,  voting).  The  set  of  queries  to  which  users  could 
respond  in  the  Polls  feature  was  fixed  —  i.e.,  responses  could  only  be  collected  to  the  pre-packaged 
probe  questions.  Owing  to  its  highly  graphic  interface,  Co-Motion  was  initially  seen  as  an 
attractive  medium  for  discussion  of  complex  issues  (e.g.,  design  tradeoffs).  As  time  went  on,  the 
relative  inflexibility  of  the  package  resulted  in  a  reduced  estimate  of  its  value.  In  other  words,  the 
provision  of  novel  collaborative  affordances,  no  matter  how  effectively  done,  was  not 
continuously  recognized  as  a  significant  achievement. 

The  examples  of  Aspects  and  Co-Motion  illustrate  a  sort  of  tradeoff  in  groupware 
implementation.  The  dimensions  of  this  tradeoff  concern  the  individual  versus  the  collective,  the 
single  workstation  versus  an  entire  network,  and  task  specialization  versus  general  functionality. 
We  say  "a  sort  of  tradeoff  because  it's  unclear  whether  these  three  dimensions  can  be  sorted  out 
into  (e.g.)  three  separate  tradeoff  specifications  or  combined  into  a  single  one.  It  is  definitely  clear 
that  these  dimensions'  relative  influence  may  vary  depending  on  situational  factors.  Both  cases 
illustrate  a  tendency  for  initial  assessment  to  concentrate  on  the  way  software  supports  an 
individual  user  at  his/her  own  workstation.  In  the  case  of  Aspects,  this  leads  to  underestimating 
the  novelty  of  the  product;  in  the  case  of  Co-Motion,  it  leads  to  overestimating  that  same  novelty. 
Both  cases  illustrate  a  concomitant  tendency  to  initially  undervalue  or  overlook  the  product's 
relative  scope  of  application  either  (1)  across  diverse  synchronous  conditions  or  (2)  over  time.  Our 
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opinion  of  the  more  basic  tool  (Aspects)  rose  with  sustained  usage,  while  our  opinion  of  the  more 
specialized  tool  (Co-Motion)  correspondingly  declined.  Too  little  is  yet  known  of  how  people 
collaborate,  so  it  is  risky  to  generally  implement  the  sort  of  prescriptive  normalization  imposed  by 
applications  such  as  Co-Motion.  For  initial  or  underspecified  collaborative  tasks,  we  believe  that 
providing  basic  tools  for  ;;;eneric  tasks  (e.g..  Aspects)  is  a  wiser  path  to  take. 

Our  opinion  is  entirely  consistent  with  recent  CSCW  research.  It  has  been  demonstrated  that 
simple  tools  with  general  functionality  can  be  remarkably  effective  at  facilitating  collaboration 
(Olson  et  ai,  1992;  Hymes  &  Olson,  1992).  By  the  same  token,  critical  analyses  of  groupware 
products’  shortcomings  have  often  cited  constraints  deriving  from  narrow  adherence  to  a 
specialized  task  model  (e.g.,  Whitaker,  1994;  Carasik  &  Grantham,  1988).  The  dangers  of  such 
overspecialization  become  apparent  when  a  groupware  product  either  fails  to  give  users  sufficient 
leeway  to  reconcile  achievable  actions  against  specifications  or  to  subsequently  modify  positions  in 
response  to  collaborators'  feedback.  The  former  effect  represents  a  failure  to  deal  with  the  CSCW- 
specific  issue  of  articulation  work  —  the  translation  of  formal  specifications  for  goals  and  work 
processes  into  feasible  results  (Robinson,  1991;  Gerson  &  Star,  1986).  The  latter  represents  a 
failure  to  deal  with  the  CSCW-specific  issue  of  mutual  influence  —  the  ongoing  flux  in  negotiating 
joint  results  (Robinson,  1991).  Because  Aspects  provides  only  a  shared  medium  for  open 
unstructured  collaboration,  it  allows  room  for  both  articulation  work  and  mutual  influence. 
Because  Co-Motion  embodies  a  particular  discursive  model  and  a  fixed  set  of  tools,  it  hinders 
articulation  work  (with  respect  to  the  discussion  itself).  Because  it  forbids  retraction  of  conunents, 
Co-Motion  cannot  allow  for  the  unfolding  of  mutual  influence. 

Desktop  Video  Conferencing 

In  October  1994,  the  CDT  Lab  was  assigned  to  explore  and  evaluate  the  feasibility  of  installing 
a  desktop  video  conferencing  (DVC)  system  within  Armstrong  Laboratory's  Human  Engineering 
Division.  Our  target  scenario  was  the  interconnection  of  approximately  6-8  senior  managers 
through  a  video  conferencing  network  available  to  each  of  them  at  the  desktop.  The  paradigmatic 
example  of  the  sort  of  product  expected  was  a  digital  video  package  providing  video  images  on  the 
desktop  computer  monitor  and  routing  video  traffic  through  the  same  LAN  as  the  computers'  own 
data  traffic.  The  expected  user  population  was  not  precisely  enumerated  at  the  beginning,  but  it 
was  eventually  specified  to  be  6  in  number,  all  of  whom  were  Macintosh  users.  After  a  massive 
compilation  of  relevant  literature,  vendor  documentation,  and  other  information  (e.g.,  from  Usenet 
news  groups.  World  Wide  Web),  we  reviewed  the  state  of  the  market  in  this  technology  area.  Our 
conclusion  was  that  this  period  (fourth  quarter  1994  /  first  quarter  1995)  was  a  particularly  risky 
time  to  be  investing  in  DVC  technology.  The  major  risks  derived  from  the  high  cost  for  decent 
DVC  infrastructure,  disparities  in  DVC  product  offerings,  and  the  fact  that  major  changes  in  the 
DVC  area  were  already  in  motion. 

We  found  that  for  microcomputer-accessible  DVC  capabilities,  the  costs  typically  ranged  from 
$1000  per  desktop  up  to  around  $5(XX)  per  desktop.  At  the  bottom  end  of  this  scale  were 
hardware-dependent  packages  utilizing  vendor-specific  capacities  and  providing  image  streams  of 
no  more  than  10-15  frames  per  second  at  best.  Such  packages  were  tjqiically  limited  to  only  one 
desktop  platform  (e.g.,  Macintosh  versus  MS-DOS),  and  they  made  little  or  no  provision  for 
interoperability  with  platforms,  communications  channels,  or  video  standards  outside  those 
incorporated  in  their  own  product.  At  the  top  end  of  the  scale  were  relatively  hardware- 
independent  packages  tying  desktop  workstations  into  what  were  effectively  dedicated  video  LANs 
operating  in  parallel  to  the  computers'  data  LANs.  These  packages  permitted  full-motion  image 
streams  (i.e.,  30  frames  per  second),  interoperability  across  platforms,  and  compatibility  with 
wide-area  communicatior  s  infrastructures  (e.g.,  ISDN). 

In  addition  to  these  commercial  products,  we  explored  the  possibility  for  using  AL  /  CFH's 
resident  expertise  to  custom-build  a  DVC  network.  This  option  was  ruled  out  fairly  quickly.  For 


one  thing,  the  cost  of  equipment  equivalent  to  that  provided  in  the  commercial  packages  was  not 
significantly  less.  For  another  thing,  we  estimated  that  the  cost  of  contract  labor  for  integrating  and 
installing  a  DVC  system  would  more  than  offset  any  savings  obtained  from  buying  generic 
equipment.  Because  the  initial  mandate  to  CDT  Lab  invoked  computer-based  video  capacities  (as 
opposed  to  parallel  anzilog  video  networks),  the  apparent  expectation  was  for  a  solution  integrated 
into  a  particular  class  of  microcomputers  —  thus  simultaneously  increasing  the  platform- 
dependency  and  decreasing  the  cost-feasibility  of  any  "homebuilt"  solution.  Finally,  any 
homebuilt  solution  would  have  had  to  rely  on  public  domain  or  shareware  support  software,  none 
of  which  we  judged  sufficiently  stable  to  meet  our  needs. 

Our  ability  to  compare  products  was  severely  constrained  by  disparities  among  the  products, 
vendor  specification  documents,  and  evaluators'  individual  assessments  of  the  specifications  we 
were  applying.  The  most  cost-effective  microcomputer  solutions  were  typically  limited  to  either 
MS-DOS  or  Macintosh  platforms.  Schemes  for  patching  together  two  or  more  such  products 
commonly  entailed  an  expensive  intermediary  channel  (e.g.,  ISDN)  which  elevated  the  composite 
cost  to  a  level  comparable  with  the  more  expensive  dedicated  video  LAN  solutions.  Out  of  a  total 
field  of  some  35  DVC  products  we  identified  on  the  market,  our  choices  quickly  boiled  down  to  4 
microcomputer-based  produets  and  3  high-end  video  LAN  paekages. 

It  was  apparent  that  costs  were  dropping  for  DVC  products  —  particularly  the  prices  on  the 
digital  /  analog  eoder-deeoder  {codec)  units  necessary  to  operate  local  DVC  stations  over  wide-area 
communications  networks.  It  was  also  apparent  that  the  higher-end  DVC  products  were  a 
relatively  economical  solution  on  a  per-workstation  basis  in  cases  where  the  number  of 
workstations  increased  beyond  the  6  we  sought  to  support.  We  don't  mean  that  the  high-end 
products  ever  became  directly  cost-competitive  with  the  low-end  products  —  only  that  they  b^ame 
increasingly  cost-competitive  as  the  total  number  of  users  increased.  Phrased  another  way,  our 
target  population  of  6  was  not  sufficiently  big  enough  to  warrant  the  high-end  products. 

It  was  also  apparent  that  the  fourth  quarter  of  1994  was  a  bad  time  to  make  long-term  decisions 
on  DVC  technologies.  A  number  of  international  standards  had  either  been  agreed  upon  or 
compiled  for  consideration  during  1994,  of  which  the  key  ones  were; 

•  ITU-T  Recommendation  H.320,  Narrow-band  visual  telephone  systems  and  terminal 

equipment. 

•  rrU-T  Recommendation  H.261,  Video  codec  for  audiovisual  services  at  p  x  64  kbit/s. 

•  ITU-T  Draft  Recommendation  T.120,  Transmission  Protocols  for  Multimedia  Data. 

These  standards  provided  the  first  common  targets  for  DVC  functionality,  and  vendors  were 
slowly  reacting  to  the  prospect  of  having  to  operate  in  conformance  with  them.  Our  evaluations 
shifted  as  vendors  stated  (e.g.,  in  the  trade  literature)  that  they  were  or  were  not  planning  for 
standards  conformance,  f  ome  vendors  —  notably  those  who  claimed  no  plans  for  conformance  — 
were  slashing  retail  prices  during  the  period  of  our  survey.  This  made  their  products  more 
attractive  in  terms  of  price  but  potentially  less  reliable  in  terms  of  longevity  in  service. 

We  recommend  that  DVC  installation  not  be  attempted  until  a  thorough  task  analysis  is 
conducted.  As  an  intervention  into  the  communicational  network,  DVC  implementation  involves  a 
very  significant  social  aspect.  We  suggest  that  a  relatively  low-cost  way  for  surveying  DVC 
opportunities  would  be  to  give  workers  small  video  telephone  units  ($899  each  from  AT&T;  $799 
each  from  MCI  as  of  December  1994)  for  trial  usage.  Compared  to  deploying  current  DVC 
technologies,  this  would  be  an  easy,  portable  testbed  compatible  with  the  existing  office  telephone 
infrastructure.  Such  a  modest  testbed  should  be  capable  of  generating  data  affirming  or  ruling  out 
options  for  further,  more  costly,  DVC  implementation. 

The  only  consistent  utility  for  DVC  identified  to  date  has  been  in  one-on-one  informal 
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conversations.  This  does  not  mean  DVC  should  be  prescribed  for  such  interactions;  DVC 
installations  to  date  have  been  plagued  by  user  misanalogies  between  video-mediated  and  face-to- 
face  conversation.  Experience  suggests  that  DVC  is  not  well-suited  for  referential  (topic-directed) 
conversation,  nor  is  conversation  easily  or  effectively  regulated  in  DVC  channels.  Although 
informal  DVC  interactions  contribute  to  social  bonding,  they  also  permit  deception  to  a  greater 
degree  than  face-to-face  interaction  (Fish  &  Kraut,  1994). 

All  in  all,  the  evidence  supports  a  view  that  video  is  not  as  important  as  either  audio  or  shared 
information  (e.g.,  graphics)  in  providing  effective  support  for  distributed  work  (Gold,  1992;  Tang 
&  Isaacs,  1992).  Although  interest  in  DVC  technologies  is  widespread,  and  naarket  offerings  are 
proliferating,  we  can  find  no  compelling  evidence  in  the  relevant  research  literature  for  video 
adding  value  to  an  established  communication  capability.  The  latest  research  results  of  which  we 
are  aware  come  from  a  study  to  be  presented  at  CHr95  (Denver,  May  1995),  in  which  there  was  a 
discernible  but  statistically  insignificant  performance  advantage  noted  for  video-enhanced 
communications  (Olson,  1995).  Weak  though  it  is,  this  would  be  among  the  first  indications  of 
value  added  by  DVC  systems. 

Our  conclusion  is  that  a  "wait  and  see"  attitude  is  still  justified  after  more  than  30  years' 
commercial  marketing  of  video  telephony  and  DVC  products.  The  DVC  market  is  changing 
significantly,  as  evidenced  by  ongoing  progress  in  standardization,  interoperability,  and 
performance.  We  see  the  immediate  future  as  a  watershed,  and  currently  reasonable  market 
choices  are  not  guaranteed  to  remain  reasonable  for  6  months  or  longer.  To  guarantee 
conformance  to  the  emerging  standards  would  require  either  (1)  buying  now  from  low-end 
vendors  and  trusting  them  to  migrate  to  full  conformance  or  (2)  buying  from  high-end  vendors 
already  in  conformance.  We  judged  the  first  (low-end)  option  to  be  unwise.  The  latter  (high-end) 
option  would  be  justified  only  to  the  extent  that  (1)  a  DVC  capability  is  needed  immediately;  (2) 
there  is  a  commitment  to  the  product  selected  as  the  platform  for  all  prospective  expansion  of  the 
DVC  network;  and  (3)  there  is  a  commitment  to  global  interoperability  for  the  DVC  network. 

The  World  Wide  Web:  Hypertext  for  Collaboration 

The  World  Wide  Web  (WWW,  W3)  project  has  been  officially  described  as  a  "wide-area 
hypermedia  information  retrieval  initiative  aiming  to  give  universal  access  to  a  large  universe  of 
documents"  (Hughes,  1994).  Originally  created  as  an  in-house  tool  for  researchers  and 
collaborators  at  CERN  (i.e.,  European  Particle  Physics  Laboratory),  the  WWW  has  grown  and 
spread  to  virtually  all  comers  of  the  globe.  Since  its  inception  in  March  1989,  an  estimated  10,000 
WWW  servers  have  been  placed  on-line  with  hundreds  being  added  daily.  The  WWW  makes 
large  amounts  of  diverse  information  (e.g.,  text,  sound,  images,  and  animated  video)  available  to 
individuals  with  Internet  access  and  the  appropriate  browser  /  interface  software.  WWW 
documents  are  formatted  using  the  Hypertext  Mark  up  Language  (HTML)  -  a  structured 
annotation  syntax  derived  from  the  Standard  Graphic  Markup  Language  (SGML).  HTML  extends 
SGML  by  providing  for  hypertext  presentation  -  i.e.,  interlinking  documents  for  free-form  user 
navigation.  WWW  documentation  constitutes  "multimedia,"  in  that  it  may  contain  text,  graphic 
images,  animation  sequences,  and  sounds.  The  ease  with  which  diverse  information  can  be 
retrieved  and  transported  has  placed  the  WWW  in  the  forefront  of  Internet  technologies.  The 
explosively  growing  population  of  WWW  server  sites  and  users  has  placed  WWW  in  the  forefront 
of  Internet  services. 

The  WWW  functions  as  a  wide-area  client-server  architecture.  Hypertext  formatted  files  are 
stored  on  a  computer  that  functions  as  an  interactive  repository,  or  server.  When  the  server 
receives  a  request  from  a  user's  browser  software  (e.g.,  NCSA  Mosaic),  or  client,  it  responds  by 
transmitting  the  requested  HTML  document  to  the  client.  Upon  receipt,  the  client's  browser 
software  converts  the  HTML  document  into  a  form  displayed  at  the  user  interface.  Conversion  and 
display  of  some  of  the  diverse  data  types  transmissible  in  WWW  (e.g.,  sounds,  color  graphics)  are 
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handled  by  auxiliaiy  applications  (other  than  the  actual  browser)  which  must  be  resident  on  the 
client  machine.  In  this  type  of  architecture,  the  server  "hands  off  files  as  demanded  by  clients, 
rather  than  (e.g.)  maintaining  an  open  connection  through  which  the  client  may  browse.  This 
arrangement  allows  the  server  to  remain  maximally  available  for  many  users'  requests.  The 
tradeoffs  are  that  compared  to  remote  log-on  and  terminal  emulation:  (1)  the  client  workstation 
must  bear  a  more  significant  burden;  (2)  the  network  traffic  between  client  and  server  will  typically 
be  higher;  and  (3)  client  response  time  (from  the  user's  perspective)  will  typically  be  slower.  A 
more  complete  description  of  the  WWW  functions  and  capabilities  is  beyond  the  scope  of  this 
document. 

In  an  effort  to  investigate  the  utility  of  the  WWW  for  AL  /  CFH  and  multimedia  /  groupware 
applications,  we  established  a  demonstration  WWW  server  on  a  Macintosh  II  computer  using 
MacHTTP  2.0  server  software.  Our  development  plan  was  divided  into  two  stages.  In  Stage  1 , 
we  developed  multimedia  materials  exclusively  for  our  Collaborative  Design  Technology 
Laboratory  to  demonstrate  the  types  /  structure  of  information  that  could  be  presented  by  other  AL  / 
CFH  components.  During  Stage  1,  access  to  the  demonstration  materials  was  limited  to  Division 
personnel.  The  demonsLation  server  was  to  be  evaluated  by  CDT  Lab  based  on  feedback  from 
Division  personnel,  in  pr;:paration  for  generating  guidelines  for  AL  /  CFH  Division  multimedia 
materials  construction. 

In  Stage  2,  we  would  apply  these  guidelines  to  demonstrate  how  information  about  all  AL  / 
CFH  Branches  and  Laboratories  could  be  presented  to  "The  World."  In  Stage  2,  CDT  Lab 
personnel  would  coordinate  /  supervise  the  HTML  formatting  and  editing  of  materials  supplied  by 
Division,  Branch,  and  Laboratory  government  personnel.  At  the  completion  of  Stage  2,  all 
materials  would  be  available  for  appropriate  AL  /  CFH  Division  personnel  for  initial  public 
deployment  on  an  established  (i.e.,  non-experimental)  WWW  server.  Planning  is  currently 
underway  within  AL  /  CF  (the  directorate  subsuming  AL  /  CFH)  for  installation  and  maintenance 
of  a  WWW  Server. 

The  Application  of  WWW  Infrastructure  to  Collaboration.  There  are  a  number  of  efforts 
underway  to  explore  how  the  HT^  and  client  /  server  features  of  the  World  Wide  Web  might  be 
used  to  support  on-line,  worldwide  collaboration.  These  are  termed  annotation  systems,  in  that 
they  permit  users  to  add  comments  to,  but  not  directly  modify,  documents  presented  via  WWW.  A 
public  annotation  system  is  one  whereby  people  from  all  over  the  Web  can  publish  their  comments 
^d  annotations  to  an  original  document  without  having  to  get  the  document's  owner  directly 
involved  (Gramlich,  199 -L  On  an  experimental  basis,  NCSA  Mosaic  version  1.2  (released  in 
1993)  supported  a  limited-range  version  of  public  annotation  support  called  group  annotations, 
allowing  annotations  to  documents  anywhere  on  a  specific  network  (e.g.,  a  LAN)  shared  by 
members  of  a  particular  workgroup.  Workgroups  would  accomplish  this  by  running  a  dedicated 
annotation  server.  Each  time  someone  accesses  a  document  anywhere  on  the  Internet,  the 
annotation  server  is  queried  for  annotations  by  any  other  workgroup  members.  If  such  annotations 
have  occurred,  hyperlinks  to  those  annotations  are  interleaved  (by  the  annotation  server)  into  the 
current  document  text.  At  the  time  of  this  writing  (April  1995),  plans  were  in  place  for  NCSA 
Mosaic  version  2.0  (then  in  beta  version)  to  eventudly  incorporate  annotation  features,  but  not  in 
its  initial  release. 

Multiple  research  groups  are  experimenting  with  mdimentary  collaboration  platforms  based  on 
HTML  annotation.  Examples  include: 

•  ComMentor  at  Stanford  University  (Rbscheisen,  Mogensen,  &  Winograd,  1995) 

•  CoNote  at  Cornell  University  (Davis  &  Huttenlocher,  1994) 

•  The  Discuss->WWW  Gateway  at  MIT  (Dvomik,  1994) 

•  The  WIT  (W3  Interactive  Talk)  structured  conferencing  system  at  CERN  in  Switzerland 
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•  The  M.I.T.  COMLINK  System  allowing  Federal  workers  to  comment  on  the  Vice 

President's  National  Performance  Review 

•  The  Mole  system  at  the  University  of  Strathclyde  (UK)  (Whittington,  1994) 

•  The  W3  Document  Annotator  at  the  University  of  Lausanne  (Switzerland)  (Schenck, 

1994) 

Each  of  these  prototypes  uses  the  HTML  /  WWW  infrastructure  to  implement  collaborative 
messaging  and  conferencing  applications.  Even  more  interesting  is  the  fact  that  the  CERN  system 
WIT  is  itself  an  instantiation  of  the  IBIS  model  previously  discussed  with  respect  to  design 
rationale.  We  believe  that  these  exploratory  efforts  are  the  breeding  ground  for  the  next  generation 
of  groupware  developments.  By  "next  generation,"  we  do  not  mean  to  imply  that  these 
prototypes'  functionality  necessarily  exceeds  current  groupware  products.  The  innovation  in  this 
case  lies  not  in  the  fin^  functionality  delivered  to  the  end  user,  but  in  the  ubiquity  and  vendor- 
independence  of  the  infrastructure  upon  which  they  are  based.  To  date,  compatibility  and 
interoperability  constraints  have  been  the  primary  banes  of  groupware  installations.  These 
constraints  have  been  aggravated,  and  in  some  cases  induced,  by  vendor-specific  features.  The 
appearance  and  dissemination  of  collaborative  infrastructure  products  based  on  international 
standards,  using  international  networks,  and  employing  widely  available  client  software,  will  likely 
have  a  impact  on  the  course  of  groupware  marketing  much  greater  than  an  innovation  in  this  or  that 
specific  functional  feature. 

Group  Interface  (GD 

There  have  been  some  attempts  to  address  what  human-computer  interface  issues  are  relevant  in 
group  applications  (e.g.,  Brooke,  1993;  Hewitt  &  Gilbert,  1993).  Such  discussions  typically 
concentrate  on  the  human-computer  interface  issues  pertaining  to  each  of  the  multiple  individu^ 
users  of  a  groupware  system  or  those  features  of  group  work  (e.g.,  turn-taking)  which  must  be 
addressed  in  configuring  the  software  employed  via  such  interfaces.  The  companion  report 
Whitaker,  Longinow  and  McNeese  (in  press)  summarizes  another  Collaborative  Design 
Technology  Laboratory  effort  exploring  what  it  would  mean  to  directly  interface  the  technology 
with  entire  groups.  This  work  has  addressed  the  human  factors  aspects  of  supporting  groups  with 
information  technology  by  exploring  novel  ways  of  configuring  the  technology  to  better  meet  the 
needs  of  teams  operating  as  teams  (as  opposed  to  operating  as  collections  of  individual  end  users). 
We  have  outlined  the  relevant  background  issues  and  state  of  the  art  in  group  IT  support,  then 
analyzed  what  that  state  of  the  art  represents.  Based  on  our  analysis,  we  have  laid  out  a  research 
strategy  reversing  what  we  consider  a  "backward"  tendency  in  prior  HCI  efforts  to  support 
complex  interactions  in  co-located  teams  with  technological  configurations  initially  developed  for 
limited  bandwidth  distributed  messaging.  By  concentrating  on  concrete  engagements  between 
users  and  IT,  we  have  generated  an  analytical  framework  appropriate  to  the  issues  critical  in 
defining  a  Group  Interface  (GI)  and  applied  that  framework  to  delineate  the  necessary  design 
tradeoffs  in  constructing  a  GI  artifact.  We  then  describe  our  initial  1994  prototype  of  a  Group 
Interface  artifact  —  the  Unified  Interface  Surface  or  UIS  —  and  discuss  the  results  of  our  usability 
evaluation  of  this  UIS  prototype. 
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CONCLUSIONS 


During  the  period  1993  to  1995,  the  Collaborative  Design  Technology  Laboratory  has  explored 
selected  issues  in  Collaborative  Design  and  Collaborative  Technology,  toward  the  end  goal  of 
innovation  in  Design  Technology.  We  would  like  to  close  by  summarizing  some  of  the  more 
general  conclusions  we  have  drawn  from  this  experience.  These  conclusions  are  "more  general"  in 
the  sense  that  they  do  not  derive  from  only  one  or  another  of  the  research  efforts  described  in  this 
report. 

Developing  "ecologically  valid"  frameworks  for  experimental  (or  other  structured)  studies  of 
collaborative  design  is  difficult.  However,  we  feel  that  solid  data  from  real-world  design  activities 
and  practicing  design  experts  can  be  collected  and  collated  in  a  sufficiently  structured  fashion  to 
provide  a  foundation  for  such  work.  Three  of  our  projects  --  the  TRACE  experimental  paradigm, 
the  ASC  Collaboration  Study,  and  the  AutoMate  paradigm  —  drew  upon  a  single  body  of  data 
collected  from  design  professionals  coneeming  an  automobile  navigation  system  task.  The 
multiple  payoffs  from  this  single  effort  have  justified  the  initially  large  investment  in  time  and 
effort.  We  believe  that  this  investment  will  be  further  justified  as  the  results  are  fed  forward  into 
one  or  more  series  of  design  studies.  We  believe  that  our  strategy  of  feeding  detailed  empirical 
data  on  design  experiences  into  TRACE  and  AutoMate,  then  making  those  paradigms  (or  sessions 
based  on  them)  available  for  inspeetion  and  feedback  by  design  professionals  (e.g.,  the  debriefings 
after  eaeh  TRACE  session)  allows  us  to  expedite  ongoing  collection  and  validation  of  ecologically 
valid  data  on  design  eollaboration. 

Collecting  observational  data  on  real-world  design  activities  is  time-consuming,  resource¬ 
intensive  work.  On  the  other  hand,  there  is  no  other  method  guaranteed  to  provide  so  detailed  a 
trace  of  actual  design  collaborations.  Barring  the  utilization  of  comprehensive  automatic  workplace 
monitoring,  there  is  no  way  to  avoid  dedicating  hundreds  or  thousands  of  person-hours  to  such  an 
effort.  Even  when  using  electronie  monitoring  (e.g.,  videotaping),  transcription  of  the  raw  data 
into  analyzable  /  archivable  form  can  entail  an  effort  of  similar  magnitude  to  the  on-site  data 
collection.  The  scale  of  investment  necessary  to  conduct  an  observational  study  should  motivate 
researchers  to  comprehensively  plan  their  tacties  and  budget  their  resources.  This  is  made  more 
critical  by  the  fact  that  observational  studies  are  essentially  "all  or  nothing"  in  nature  —  once  begun, 
they  must  be  carried  through  to  eompletion.  Our  experience  affirms  the  utility  of  observational 
approaehes  in  studying  design  eollaboration.  Recent  advocaey  of  such  methods  (partieularly  in 
academic  circles)  is  well-justified,  but  solid  "how-to"  information  and  /  or  experienced 
observational  researchers  are  less  readily  obtained.  We  are  concerned  that  for  large-scale,  mission 
critical  design  exereises  (as  are  typieal  in  USAF  design  acquisition),  researchers  (or  others  —  e.g., 
quality  managers)  contemplating  observational  data  collection  should  carefully  assess  the  eosts  and 
requirements  of  doing  sucn  work  effectively. 

Collaborative  teehnologies  are  (as  unit  investments)  more  expensive  than  earlier  types  of  IT 
products.  We  do  not  mean  to  imply  that  groupware  supporting  X  workers  neeessarily  costs  more 
than  (e.g.)  individual  productivity  packages  for  each  of  those  X  workers.  We  only  mean  to 
emphasize  that  the  unit  investment  entailed  in  a  single  groupware  innovation  necessarily  includes 
the  cost  of  innovation  covering  all  X  of  those  workers  at  once.  For  a  LAN-based  system,  there 
must  be  sufficient  investment  to  achieve  a  parity  of  equipment  among  collaborators  in  addition  to 
the  necess^  infrastrueture  (e.g.,  LAN  cables,  servers,  etc.).  Compatibility  and  interoperability 
issues  require  more  than  simple  lip  service,  beeause  everyone  must  be  able  to  work  together.  The 
high  scale  of  basic  investment  is  matched  by  a  correspondingly  high  level  of  enterprise 
commitment  required  to  get  everything  "up  and  running"  and  everyone  "on  board."  These  factors 
result  in  groupware  being  a  "high  stakes"  investment  whose  risks,  like  its  advertised  payoffs,  are 
of  a  strategic  scale. 

Collaboration  is  not  simply  a  matter  of  installing  eollaborative  teehnology.  Reeall  our  earlier 
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contrast  between  two  models  of  productive  collaboration:  that  of  Ford  (linear  production  via 
individual  throughput  on  subtasks)  versus  Volvo  /  Kalmar  (flexible  production  via  team  synergy 
throughout  the  whole  task).  Concurrent  engineering,  the  USAF's  IPD,  and  other  n^cent 
innovations  in  design  metnodology  represent  a  shift  from  the  Ford  to  the  Kalmar  style.  This  is  not 
something  that  can  be  implemented  through  simple  invocation  of  change.  Today's  design 
professionals  are  accustomed  to  linear,  compartmentalized  design  paths  along  which  they  feed 
forward  their  results  by  "tossing  it  over  the  wall"  to  the  next  person.  Changing  habits  and  attimdes 
from  the  "Ford  era"  will  take  time,  and  attention  must  be  paid  to  the  social  /  behavioral  impacts  of 
innovations  entailing  collaborative  practices. 

Collaborative  technologies  cannot  reasonably  be  evaluated  using  the  same  techniques  or  metrics 
as  prior  IT  innovations.  Assessing  group  productivity  or  collaborMion  payoff  is  not  as 
straightforward  as  measuring  (e.g.)  individu^  throughput.  Combined  with  the  necessarily  large 
costs  and/or  risks  for  groupware  implementation,  this  makes  it  difficult  for  management  and 
planners  to  prospectively  justify  migration  to  collaborative  IT  solutions.  Nonetheless,  there  are 
some  conventionally-framed  results  (in  terms  of  Return  on  Investment  ox  ROT)  which  can  be  cited. 
In  a  1994  survey  of  65  Lotus  Notes™  users.  International  Data  Corporation  found  an  average  3- 
year  ROI  of  179%  (of  investment)  and  an  average  investment  recoupment  period  on  the  order  of 
2.4  years  (Simpson,  1994).  The  average  initial  investment  for  each  subject  organization's 
enterprise-wide  Notes  implementation  was  $240,000.  Because  Notes  is  more  an  "infrastmcture" 
than  an  "application"  product,  we  believe  this  scale  of  payoff  is  probably  higher  and  more  clearly 
delineated  than  would  be  the  case  for  more  specialized  groupware  acquisitions  (e.g.,  group 
decision  support  systems).  As  such,  we  see  no  reason  to  predict  that  assessing  groupware  payoffs 
will  become  easier  in  the  foreseeable  future. 

The  aforementioned  joint  levels  of  investment  and  commitment  for  groupware  are  necessitated 
by  the  fact  that  IT-supported  collaboration  entails  conformity  and  compatibility.  If  one  considers 
groupware  at  the  scale  of  global  (as  opposed  to  enterprise-specific)  conformity  and  compatibility,  it 
becomes  apparent  that  one  or  another  user  organization  is  not  likely  to  exert  sufficient  leverage  to 
steer  marketplace  developments.  It  will  become  increasingly  difficult  to  justify  in-house  products 
by  comparison  with  commercial  offerings.  This  effect  will  be  compounded  by  the  integration  of 
internationally-recognized  standards  into  the  requirements  specifications  for  all  products  and  the 
activities  they  support.  This  is  not  necessarily  negative  -  the  arrival  of  international  standards 
(e.g.,  the  H.320  suite)  and  integrated  tools  (e.g.,  Apple  MovieTalk™)  for  desktop  video 
conferencing  will  certainly  result  in  decreased  unit  costs  for  DVC  applications.  On  the  other  hand, 
this  is  not  to  say  that  the  future  marketplace  will  always  keep  its  eyes  solely  on  the  future.  The 
"irresistible  force"  of  explosive  growth  for  infrastructure  products  such  as  die  World  Wide  Web 
may  well  coalesce  into  "immovaWe  objects"  with  which  subsequent  innovators  must  interoperate, 
regardless  of  any  appearance  of  better  options  in  the  mean  time.  As  a  result,  we  see  no  reason  for 
considering  in-house  development  of  groupware  "from  the  ground  up"  except  for  research,  highly 
specialized,  or  mission-critical  applications. 
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Pointers  to  Relevant  Resources  in  the  World  Wide  Web  (WWW) 


56 


The  following  are  selected  addresses  for  relevant  information  resources  on  the  World 
Wide  Web  (WWW).  The  items  beginning  with  "http://..."  are  Uniform  Resource 
Locators  (URLs)  —  the  standard  addressing  convention  for  WWW. 

I.  INFORMATION  ABOUT  THE  WORLD  WIDE  WEB 

A.  World  Wide  Web  Development: 

1.  Virtual  Library/Cyberweb:  WWW  Development 
http://www.charm.net/~web/Vlib.html 

2.  Stanford’s  Yahoo  Server:  Computers/WWW 
http://akebono.stanford.edu/yahoo/Computers/World_Wide_Web 

3.  CERN  (World  Wide  Web  Originators) 
http://info.cem.ch 

4.  Massachusetts  Institute  of  Technology  (WWW  Host  US  Sponsor) 
http://web.mit.edu 

5.  INRIA  (W  WW  Host  European  Sponsor) 
http://www.inria.fr 

B.  Search  Engines  (i.e.,  tools  for  searching  WebSpace) 

1.  Web  Crawler 

http://webcrawler.cs.washington.edu/WebCrawler/Home.html 

2.  WWW  Worm 

http://www.cs.colorado.edu/home/mcbryan/WWWW.html 

3.  Lycos 

http://lycos.cs.cmu.edu 
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C.  WWW  Catalogs  (some  of  which  provided  automated  search 


capabilities) 

1.  CERN’s  Virtual  Library 

http://info.cem.ch/hypertext/DataSources/bySubject/Overview.html 

2.  Whole  Internet  Catalog  at  O’Reilly  and  Associates 
http://nearaet.gnn.com/wic/newrescat.toc.html 

3.  CERN 

http://cuiwww.unige.ch 

4.  Stanford’s  Yahoo 
http://akebono.stanford.edu/yahoo 

5.  EINet  Galaxy 
http://www.einet.net 

6.  University  of  Michigan  Clearinghouse 
http  ://www  .lib.  umich.edu/chhome.html 

7.  Planet  Earth 

http://teal.nosc.mil/info.html  “Planet  Earth” 

8.  Yanoff  s  Connections 
http://www.uwm.edu/Mirror/inet.services.html 

11.  CSCW,  HUMAN  FACTORS  AND  RELATED  WEB  SERVERS 

A.  Human-Computer  Interaction 

1.  Index  of  HCI  Related  Information 
http://www.twi.tudelft.nl/Local/HCI/HCI-Index.html 

2.  HCI  section  of  the  WWW  Virtual  Library 
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http://www.cs.bgsu.edu/HCI/ 


3.  HCI  Resources 

http://www.ida.liu.se/labs/aslab/groups/uin/hci 

B.  Computer  Supported  Cooperative  Work 

1.  CSCW  Directory 

http://www.demon.co.uk/jrac/cscwdir.htinl 

2.  The  Unofficial  Yellow  Pages  for  CSCW 
http://www.tft.tele.no/cscw/ 

3.  Web-Searchable  Bibliography  on  CSCW 

http://wwwl  1  .informatik.tu-muenchen.de/cscw/cscw-biblio.html 

4.  Collaborative  Software  Resource  Clearinghouse 
http  ://w  ww  .ics  .hawaii  .edu/~jl/CSRC.htnil 

C.  Desktop  Video  Conferencing  (DVC) 

1.  Introduction  to  DVC 
http://fiddle.ee.vt.edu/succeed/videoconf.html 

2.  Survey  of  DVC  products 

http://www2.ncsu.edu/eos/service/ece/project/succeed_info/dtvc_survey/prod 

ucts.html 

3.  The  H.320  video  telephony  standards  documentation  (from  ITU) 
http://www.itu.ch/itudoc/metadocs/23397.html  (direct  download  of  H.320) 
gopher ://info.itu.ch/l  l/.l/Stds-Pub-etc  (Gopher  access  to  all  ITU  standards) 
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D.  Computer-Mediated  Communication 


1.  Computer-Mediated  Communication  Studies  Center 
http://www.rpi.edu/~decemj/cmc/center.html 

2.  The  Internet  and  Computer-Mediated  Communication 
ftp://ftp.rpi.edu/pub/communications/intemet-cmc.html 

E.  Other  Relevant  Resources 

1 .  American  Communication  Association 
http://cavem.uark.edu/comniinfo/www/ACA.html 

2.  Information  Science  World 
http://www.cox.smu.edu/mis/iswnet/home.html 

3.  National  Coordination  Office  /  High  Performance  Computing  and 
Communications 

http://www.hpcc.gov 

4.  Misc.creativity  home  page  (includes  some  material  on  group  facilitation) 
http://www.unidata.coni/~ucc01/creative.htm 
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APPENDIX  B 


Published  Periodicals  of  Relevance  to  CSCW 
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Computer  Supported  Cooperative  Work:  An  International  Journal 
Collaborative  Computing 
ACM  SIGCHI  Bulletin 
ACM  SIGOIS  Bulletin 

ACM  Transactions  on  Computer-Human  Interaction 

ACM  Transactions  on  Office  Information  Systems 

Communications  of  the  ACM 

Decision  Systems 

Group  Decision  and  Negotiation 

Human  Computer  Interaction 

International  Journal  of  Man-Machine  Studies 

International  Journal  on  Intelligent  and  Cooperative  Information  Systems 
Management  Communication  Quarterly 
MIS  Quarterly 

Office:  Technology  and  People 
Organizational  Science 
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APPENDIX  C 


Usenet  Newsgroups  of  Relevance  to  CSCW  and  other  Topics  Discussed  in  this 

Technical  Report 
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bit.listserv.quality 

bit.listserv.qualrs-l 

comp.cog-eng 

comp.dcom.* 

comp.groupware 

comp.groupware.lotus-notes.misc 
comp,  human-factors 
comp.infosystems.www.* 


(quality  management,  TQM,  etc.) 

(qualitative  research,  observational  studies) 
(cognitive  engineering) 

(hierarchy  of  groups  about  data  communications) 
(CSCW  and  groupware) 

(Lotus  Notes) 

(human  factors  and  HCI) 

(hierarchy  of  groups  about  the  World  Wide  Web  — 


WWW) 


comp.multimedia 

misc. business .  facility  vors 

rec.video.desktop 

sci.anthropology 

sci.cognitive 

sci.med.  telemedicine 


(multimedia) 

(meeting  /  group  process  facilitation) 

(desktop  video  production  and  some  DVC  issues) 
(occasional  news  and  information  on  ethnography) 
(cognitive  studies) 

(distributed  medical  applications) 
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